science and technology

My glamorous life: are you ready to math?

For the past two years, I’ve been publishing a daily work-and-life diary on Basecamp, sharing it with a few friends. This private writing work supplanted the daily public writing I used to do here. In an experiment, I’m publishing yesterday’s diary entry here today: YESTERDAY, Ava and a few of her schoolmates participated in a […]

The post My glamorous life: are you ready to math? appeared first on Zeldman on Web & Interaction Design.




science and technology

You got this.

I’M LEARNING new tech and it’s hard. Maybe you’re in the same boat. Through the rosy lens of memory, learning HTML and Photoshop back in the day was a breeze. It wasn’t, really. And CSS, when it came along in 1996, was even tougher to grasp—in part because it was mostly theoretical, due to poor […]

The post You got this. appeared first on Zeldman on Web & Interaction Design.




science and technology

Design is a (hard) job.

DESIGN WAS so much easier before I had clients. I assigned myself projects with no requirements, no schedule, no budget, no constraints. By most definitions, what I did wasn’t even design—except that it ended up creating new things, some of which still exist on the web. Soon I had requirements, schedules, and constraints, but most […]

The post Design is a (hard) job. appeared first on Zeldman on Web & Interaction Design.





science and technology

The Beauty Trap in Design [Automattic.Design]

I love a good page layout. I’m a chump for a visually well composed series of paragraphs. The proper degree of corner rounding for a given set of photos in relation to a box three columns over sets my little heart aflutter.

The post The Beauty Trap in Design [Automattic.Design] appeared first on Zeldman on Web & Interaction Design.




science and technology

Rams

We show our audience Gary Hustwit’s “Rams”—a documentary about product design icon Dieter Rams—during the lunch hour at An Event Apart. We’ve shown Gary’s film in every city of our tour this year, and every time I’ve watched it with our attendees I’ve seen new things in the film, and been ever more deeply moved by it.

The post Rams appeared first on Zeldman on Web & Interaction Design.




science and technology

My Brunch with Jen

Jen was present for, and actively participated in, the very beginnings of the creative and blogging web, and her famous book, now in its umpteenth edition, is still the best introduction to web design I know—probably the best that will ever be written.

The post My Brunch with Jen appeared first on Zeldman on Web & Interaction Design.




science and technology

Design Kickoff Meetings

Posted here for posterity: Design kickoff meetings are like first dates that prepare you for an exciting relationship with a person who doesn’t exist.

The post Design Kickoff Meetings appeared first on Zeldman on Web & Interaction Design.




science and technology

Expressive Design Systems

Yesenia Perez-Cruz started her career as a designer at Happy Cog Philadelphia. From the first day, her design gifts were unmistakable. As her career progressed, she moved from one challenging role to another. At companies like Vox Media and Shopify, and at conferences around the world, she has been a design team leader, a popular […]

The post Expressive Design Systems appeared first on Zeldman on Web & Interaction Design.




science and technology

Let’s hang (Spotify)

Love music? Follow your own tastes? Let’s share. Connect on Spotify.Connect on Last.fm. As a bonus, if we connect on Spotify, you not only get access to An Event Apart’s playlists from the past decade, you also get a preview of the 2020 playlist in progress.

The post Let’s hang (Spotify) appeared first on Zeldman on Web & Interaction Design.




science and technology

Another Blue Beanie Day

Yesterday was the nth annual Blue Beanie Day. (I’ve lost track of what year the standardista holiday started.) I was awake at 1:00 AM on Friday night/Saturday morning, so I tweeted “Happy #BlueBeanieDay,” then slept. No blog post, no prelude—just a past-midnight tweet, over and out. Saturday, once or twice, I checked Twitter and retweeted […]

The post Another Blue Beanie Day appeared first on Zeldman on Web & Interaction Design.




science and technology

A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX.

It’s one thing to seek diverse talent to add to your team, another to retain the people you’ve hired. Why do so many folks we bring in to add depth and breadth of experience to our design and business decision-making process end up leaving? Hear thoughtful, useful answers to this question and other mysteries of […]

The post A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX. appeared first on Zeldman on Web & Interaction Design.




science and technology

The Web We Lost: Luke Dorny Redesign

Like 90s hip-hop, The Web We Lost™ retains a near-mystical hold on the hearts and minds of those who were lucky enough to be part of it. Luke Dorny’s recent, lovingly hand-carved redesign of his personal site encompasses several generations of that pioneering creative web. As such, it will repay your curiosity.

The post The Web We Lost: Luke Dorny Redesign appeared first on Zeldman on Web & Interaction Design.




science and technology

My Glamorous Life

At 4:00 PM, I went to bed to rest up from my head cold, and promptly fell asleep. When I awoke, the clock said 7:15. Oh, no! I banged on my daughter’s door. “You’re going to be late to school!” I shouted. She cackled with laughter.“It’s 7:15 AT NIGHT,” she explained.

The post My Glamorous Life appeared first on Zeldman on Web & Interaction Design.




science and technology

A kindness

Dino works six days a week as a porter in my apartment building, cleaning walls and floors, removing trash, distributing recyclables. He’s one of those essential workers who are suddenly on the front lines. We’ve always been friendly. I’ve been hibernating in my apartment for days, because it’s what we’re all supposed to do, and […]

The post A kindness appeared first on Zeldman on Web & Interaction Design.




science and technology

Going viral

Finishing Week 4 with Coronavirus, heading into Week 5. I’m home—haven’t needed to go to the hospital, thank God—and my fever petered out last week. So all that’s left are cold and cough symptoms and a totally debilitating complete lack of energy. Oh, and lower back pain: a bad cough threw one side of my […]

The post Going viral appeared first on Zeldman on Web & Interaction Design.





science and technology

The hump

Corona Virus Week 6. Recovering, bit by bit. Still get winded carrying a light package more than a dozen steps. I can sit up in the morning, make breakfast, and listen to music for several hours. Am exhausted and useless by 1:00 PM—but it’s an improvement over the 24/7 exhaustion of the previous weeks. Tomorrow […]

The post The hump appeared first on Zeldman on Web & Interaction Design.




science and technology

The Whims

One of my first professional jobs was at a tiny startup ad agency in Washington, DC. The owner was new to the business and made the mistake of hiring a college buddy as his creative director. This guy was not up to the job. He was not the slightest bit curious about our clients’ businesses, […]

The post The Whims appeared first on Zeldman on Web & Interaction Design.





science and technology

An intro to making Postgres high availability on Kubernetes

#351 — April 15, 2020

Read on the Web

Postgres Weekly

A Detailed Look at pg_show_plans — A few issues ago we linked to a basic introduction to pg_show_plans – this goes a little further. pg_show_plans lets you look at the execution plans of slow queries in real time as they’re being executed which can help you when troubleshooting.

Kaarel Moppel

Intersecting GPS Tracks to Identify Infected Individuals — I’m not a huge fan of COVID-19 related content, but this is a pretty interesting technique with numerous use cases. Essentially it uses PostGIS to identify overlapping paths.

Florian Nadler

Online Training: Learn PostgreSQL from Home — The remote PostgreSQL Database Administration training course is available at a discounted rate & will be conducted in two different timezones. The course covers day-to-day DBA operations, monitoring, server configurations, and more.

2ndQuadrant PostgreSQL Training sponsor

PostgreSQL's 'Related Projects' — Thanks to Andreas Scherbaum for pointing out a new page on the Postgres site dedicated to projects related to Postgres like the code that runs the Postgres web site, mailing list, build farm, package management system, etc.

PostgreSQL Global Development Group

Authentication Configuration in Postgres (and CockroachDB) — In Postgres, client authentication can be controlled via a ‘HBA’ (host-based authentication) file. It’s not something we see covered very often, so you might find this interesting, particularly as it compares things against CockroachDB.

Raphael ‘kena’ Poss

▶  Easy And Correct High Availability Postgres with Kubernetes — A 50 minute talk from PostgresOpen 2019 that goes all the way ‘from containers up’ until actually doing stuff with Postgres.

Steven Pousty

How To Set Up an Express API Backend Project With Postgres — A pretty extensive walkthrough of creating an HTTP API using Express with Node.js and Postgres on the backend, then deploying it all on Heroku.

Chidi Orji

A Beginners Guide to Basic Indexing in Postgres

James Bannister

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Documenting the Citus Extension to Postgres: An Interview with Joe Nelson — Joe, a.k.a. begriffs, talks about why he works on documentation, why the multi-tenant and real-time analytics tutorials matter, the INSERT..SELECT with repartitioning feature, and what development platform Citus uses for docs.

Citus Data (Microsoft)

Procedural vs Query Approaches for Finding Packages — Explorations of a query that can be used to display which packages are available for a given FreeBSD port. Get your head around the data model and the ideas here apply to all sorts of situations.

Dan Langille

???? Upcoming Events

All in-person events we had listed are cancelled or postponed due to the COVID outbreak, so we're now linking to webinars, livestreams, and similar online events.

If you have any, just hit reply and if it's Postgres related (and either free or not too expensive) we'll include it in a future issue. Just one this week:

???? – requires e-mail address or registration
???? – costs money to participate

???? Seen on Twitter

Saw this tweet and thought it was a pretty neat reminder of the sorts of things we can do with Postgres. Justin kindly let us include it:

Click through to the original tweet if you want to see the code better. Neat use for a generated column!




science and technology

The one where we interviewed Luca Ferrari

#352 — April 22, 2020

Read on the Web

???? We've got a neat bonus for you this week at the bottom of the issue ????

Postgres Weekly

Postgres Explain Visualizer 2: A Vue.js Component to Show Execution Plans — Less a standalone tool and something you’d use when building your own Postgres tooling. There is, however, a demo here. The output is really nifty.

Dalibo

Insert-Only Tables To Be Autovacuumed in Postgres 13 (But Why?) — Autovacuuming clears up dead tuples that are often left when updating or deleting data from tables, so why is autovacuuming for append-only tables a big deal in Postgres 13? Laurenz explains.

Laurenz Albe

Faster CI/CD for All Your Software Projects Using Buildkite — See how Shopify scaled from 300 to 1800 engineers while keeping their build times under 5 minutes.

Buildkite sponsor

Full Text Search in Milliseconds with Rails and Postgres — If you’ve never played with full text search with Postgres and Rails, this is a fine place to start. It covers LIKE/ILIKE, trigrams, and ‘proper’ full text searching. We also get to see how Leigh took a query from taking 130ms down to 7ms.

Leigh Halliday

An Easy Postgres 12 and pgAdmin 4 Setup with Docker — Docker provides an easy and loosely coupled way to get things set up in a development environment.

Jonathan S. Katz

Is There a Limit on Number of Partitions Handled by Postgres? — Sort of, but you’d really have to be going at it to stretch Postgres 12’s capabilities in this area.

Denish Patel

Where Do My Postgres Settings Come From? — A nice visual look at how parameters and settings cascade or override each other.

My DBA Notebook

Identify Slow-Running PostgreSQL Queries Quickly in Datadog — Improve PostgreSQL performance by visualizing and identifying errors fast using granular, out-of-the-box dashboards in Datadog.

Datadog sponsor

Replicate Multiple Postgres Servers to a Single MongoDB Server using Logical Decoding Output Plugin

David Zhang

An Overview of the JOIN Methods in Postgres

Kumar Rajeev Rastogi

???? A Q&A with…
Luca Ferrari
Postgres community organizer, author, adjunct professor, and open source advocate.

Luca Ferrari has had a huge impact on the Postgres community in Italy, having been president of the Italian PostgreSQL Users Group in the past and having helped to organize the popular PGDay.it events. He also blogs frequently about Postgres and wrote PostgreSQL 11 Server Side Programming Quick Start Guide for Packt.

Note: A more complete version of this interview is on the Web.

We caught up with him to ask about server side Postgres use cases in particular:

For those who use Postgres as a simple database and haven't touched the deeper elements, where do you think they should start?

There's no single answer to this question, since Postgres is such a huge project with so many features and a rich community. I never found a project where it cannot fit in. Postgres is somehow like Unix: you cannot touch it as "just a database", you need to commit to its culture to benefit the most out of it.

In my classes, I can see that people usually get fascinated by the capabilities of doing server side programming, and that is why I decided to write my book about this topic. Often, people do not expect to be able to embed their Perl, Java, or Python libraries directly into PostgreSQL without having to rewrite their business logic in an SQL-like language.

Another great feature nowdays is the support to JSON within the database, thanks to which PostgreSQL can be used as both a relational database and a 'NoSQL' storage engine, providing a lot of flexibility in your infrastructure.

One suggestion I always gave is to join the mailing lists: there are several that differ by topic and amount of traffic. Most are very active and have high quality contributors that take care in providing accuate replies to users' questions, that spend time in reproducing errors and edge-cases, and who will help you. That's a mandatory place where you have to start, in my opinion, to better learn about the project, its features and its culture.

Where should the line be drawn between doing things in an external programming language versus within Postgres?

Often the right choice is to place business logic near the data it refers to, that is within the database itself. However, there are several things to take into account including the developer's experience and the expressiveness of SQL-derived languages like pl/PgSQL.

There's a habit of letting ORMs (Object Relational Mappers) do most of the database interaction nowadays, reducing the database to a "simple storage". Of course, databases can do a lot more, and PostgreSQL in particular can help you migrate and embed your own business logic into the database itself.

I have helped a few companies embed their own Java libraries into Postgres resulting in a more robust and coherent way to access the data (the real value) without any regard of the application they were using. Because once you start having data, you will soon find that such data is required by multiple applications in different technologies and on different platforms, so that implementing the same business logic rules over and over becomes a huge effort; on the other hand, moving such logic within the database simplifies and keep uniform the way your data is manipulated.

What one thing do you think people should learn?

Stored procedures. They serve as a common base for triggers and are very similar to routines, therefore allowing you to build more complex pieces into your own cluster. Once you have learnt the common way of defining functions, you can go deeper and write your own native functions using other languages (e.g. C). This is more complex, but thanks to the extensibility of Postgres is not an impossible task and can help you migrating more and more code into the database. Once you have created a new feature, please do contribute it back so that other people can use it!

...

Last, allow me to announce that I'm working on another book right now: me and a friend of mine are writing a more general book on Postgres that will try to answer your question by leading the reader through the main features that make Postgres unique and great.

Be sure to check out Luca's book PostgreSQL 11 Server Side Programming Quick Start Guide to learn more about the topics covered in this interview. You can also find the code from the book in this GitHub repo.

If you enjoyed this interview, Luca actually gave some more detailed answers in the full interview which you can read here.




science and technology

Workloads, acceleration, and making Postgres better

#353 — April 29, 2020

Read on the Web

Postgres Weekly

7 Things That Could Be Improved in Postgres — As 1990s dance pop group D:Ream sang in 1994, Things Can Only Get Better.. including Postgres ???? Luckily these are all ‘nice to have’s but I dare say we’ll see some of them (such as automatic tuning and auto-vacuuming improvements) appear over time.

Kaarel Moppel

How The Citus Distributed Query Executor Adapts to a Postgres Workload — Citus is the popular extension for horizontally scaling Postgres and its query executor has seen some huge updates lately.

Citus Data (Microsoft)

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this pganalyze eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Swarm64 DA 4.0: A Database Acceleration Extension for Postgres — Swarm64 started life as a FPGA-driven way to accelerate Postgres performance, but can now work without FPGAs too. This is not a free product but if you want to give it a run, there’s a trial or it can be spun up from the AWS Marketplace.

Yana Krasteva

Postgres Performance Goalposts — An interesting heuristic from Bruce here on what to do if you expect your connections, queries, or write queries to be above/below certain levels.

Bruce Momjian

A Tale of Password Authentication Methods in Postgres“Let’s say you want to implement a password authentication method in a client/server protocol..” Here’s the story of how Postgres came up with its approaches.

Peter Eisentraut

How to Set application_name When Using psql — As Craig says: “Setting your application name in Postgres is SO USEFUL. It will help a lot for debugging when you’ve got multiple different apps/services connecting to the same database.”

Denish Patel

How to Upgrade Postgres from v11 to v12 on Ubuntu 20.04 — Now that Ubuntu 20.04 is out, this might be on your mind!

Paolo Melchiorre

Working with Amazon Aurora PostgreSQL: What Happened to the Stats? — Apparently there’s a bug with numerous versions of Aurora PostgreSQL that causes certain stats to be lost on restart.

Michael Vitale

Postgres Vision 2020 - Free Online Conference (June 23-24) — Learn how today’s IT leaders are using Postgres. Join from anywhere in the world and listen from 30+ Postgres experts.

EnterpriseDB sponsor

A Deep Dive into PostGIS Nearest Neighbor Search — Take a deep dive into the Postgres and PostGIS internals to find out how K-nearest neighbor accelerates local search.

Martin Davis

My Favorite Postgres Extensions: Part One — A basic high level look at pg_partman and postgres_fdw.

Nawaz Ahmed

Kanel: Generate TypeScript Types from Postgres

Kristian Dupont

Postgres.app: The Easiest Way to Get Started with Postgres on the Mac — I’ve used this for years, it’s super popular, but if there’s just a handful of developers out there who’d benefit from it and don’t know about it, this reminder will be worth it :-) It continues to get very frequent updates.

Jakob Egger, Chris Pastl, and Mattt Thompson

???? Upcoming Events

All in-person events we had listed are cancelled or postponed due to the COVID outbreak, so we're now linking to webinars, livestreams, and similar online events.

If you have any, just hit reply and if it's Postgres related (and either free or not too expensive) we'll include it in a future issue. Just one this week:

  • ???? Postgres Vision 2020 on June 23-24. A full attempt at an online Postgres conference across multiple days with multiple tracks.

???? – requires e-mail address or registration
???? – costs money to participate




science and technology

Medium-hard SQL questions to think about

#354 — May 6, 2020

Read on the Web

Postgres Weekly

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

Monitor Custom Postgres Metrics in Real-Time with Datadog — Monitor and visualize Postgres performance in context end-to-end alongside the rest of your stack. Create custom, drag-and-drop dashboards to quickly view analytics on any Postgres metric. Try Datadog free.

Datadog sponsor

My Favorite PostgreSQL Extensions: Part Two — The second part of a series we linked to last week. This time, Nawaz takes a look at pgAudit, pg_repack, and HypoPG.

Nawaz Ahmed

Backup Manifests and pg_verifybackup in Postgres 13 — Postgres 13 will introduce two features to enhance the automated validation of physical backups: backup manifests and a new tool called pg_verifybackup.

Gabriele Bartolini

arm64 Packages Now on apt.postgresql.org — If you’re running ARM64 hardware and Debian or Ubuntu, you can now install Postgres via apt.

Christoph Berg

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using Postgres for JSON Storage — With JSON and JSONB types and associated advanced ways to query such columns, using Postgres as a store for JSON data is pretty simple. This is the briefest of overviews but leads into an interactive online tutorial.

Steve Pousty

How to Migrate From Inheritance-Based Partitioning to Declarative Partitioning — Partitioning was introduced in Postgres 10 and Postgres 11 improved the declarative partitioning support. This article demonstrates a move from inheritance based partitioning to declarative partitioning using the native features found in Postgres 11+.

Caterina Magini

Free eBook: How to Get a 3x Performance Improvement on Your Postgres Database — Learn our best practices for optimizing Postgres query performance for customers like Atlassian and how to reduce data loaded from disk by 500x.

pganalyze sponsor

How to Backup Multiple Tablespaces with pg_basebackup

Ahsan Hadi

▶  Security and Compliance with Postgres — A recorded webinar that 2ndQuadrant ran recently.

Boriss Mejías

Oracle to PostgreSQL: ANSI Outer Join Syntax in Postgres — The latest in a series of blog posts about migrating to Postgres from Oracle which looks at what Postgres offers in place of Oracle’s join operators.

Kirk Roybal

An Interview with 2ndQuadrant's Jimmy Angelakos — The latest ‘PostgreSQL Person of the Week’ to face questions about his experiences with Postgres.

Andreas Scherbaum

dadbod.vim: A Modern Database Interface for Vim — A Vim plugin for interacting with numerous databases, including Postgres.

Tim Pope

???? Upcoming Online Events

  • Postgres Pulse - weekly at 11am ET each Monday. Weekly Zoom-based sessions with folks like Bruce Momjian, Vibhor Kumar, and other people at EnterpriseDB.
  • ???? Postgres Vision 2020 on June 23-24. A full attempt at an online Postgres conference across multiple days with multiple tracks.

???? – requires e-mail address or registration
???? – costs money to participate




science and technology

It's time to upgrade those Ruby 2.4 apps

#497 — April 16, 2020

Read on the Web

Ruby Weekly

Bye Bye Ruby 2.4, Support Has Ended — From the end of April 2019 till now, Ruby 2.4 has been in its ‘security maintenance’ phase but now you won’t even get that, Ruby 2.4.10 should be the final 2.4 release. 2.5 will follow in 2.4’s footsteps next year, so upgrading to 2.6 or 2.7 should now be a priority for those older apps.

Ruby Core Team

Testing Ruby Decorators with super_method — Have you ever wondered how you can properly test the behavior of a method overridden by Module#prepend? Enter super_method which returns a Method object of which superclass method would be called when super is used or nil if none exists.

Simone Bravo

You Hacked the Gibson? Yeah, They Built Their Own Login — Don't let Crash Override pwn your app. FusionAuth adds secure login, registration and user management to your app in minutes not months. Download our community edition for free.

FusionAuth sponsor

Heya: A Sequence Mailer for Rails — “Think of it like ActionMailer, but for timed email sequences.” Note: It’s open source but not free for commercial use beyond a certain point.

Honeybadger Industries LLC

A Final Report on Ruby Concurrency Developments — A report on work funded by a 2019 Ruby Association Grant that puts forth a proposal of using non-blocking fibers to improve Ruby’s concurrency story.

Samuel Williams

Mocking in Ruby with Minitest — Minitest has basic mocking functionality baked in, but be judicious in your use of it.

Heidar Bernhardsson

???? Jobs

Ruby Backend Developer (Austria) — We’re seeking mid-level and senior devs to join us and build top-class backend infrastructure for our adidas apps, used by millions. Our stack includes: jRuby, Sinatra, Sidekiq, MySQL, & MongoDB.

Runtastic

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

▶️ Get ready for your next role: Pluralsight is free for the entire month of April. Stay Home. Skill Up. #FreeApril — SPONSORED

???? Articles & Tutorials

Predicting the Future With Linear Regression in Ruby — Linear regression is a mathematical approach to modelling a relationship between multiple variables and is demonstrated here by exploring whether the tempo of a song predicts its popularity on Spotify.

Julie Kent

Feature Flags: A Simple Way to 'De-Stress' Production Releases — Feature flags bridge a gap between the abstract concept of continuous delivery and tactical release of features.

Matt Swanson

A Guide to Deprecation Warnings in Rails — If you’ve upgraded Rails and you start seeing warnings screaming at you, you can either get Googling or.. read this ????

Luciano Becerra

What's the Difference Between Monitoring Webhooks and Background Jobs

AppSignal sponsor

Understanding webpacker.yml — Have you ever really gone through the Webpack config?

Ross Kaffenberger

Using Optimizer Hints in Rails — Rails 6 removes the need to write raw SQL to use optimizer hints, so that’s cool.

Prateek Choudhary

Dissecting Rails Migrations — You should pick up something new about migrations by reading this article as it covers all of the essentials and a little more.

Prathamesh Sonpatki

The Basics of Custom Exception Handling — Never hurts to revise the basics of effective exceptions.

Mark Michon

How to Improve Code Readability with Closures

Andrey Koleshko

???? Code and Tools

ruby-prolog: A Pure Ruby Prolog-like DSL for Logical Programming — Solve complex logic problems on the fly using a dynamic, Prolog-like DSL inline with your normal code.

Preston Lee

Anyway Config: Keep Your Ruby Configuration Sensible — Get your Ruby project out of ‘ENV Hell’ with anyway_config, a framework for managing configuration.

Vladimir Dementyev

The End of Heroku Alerts — Rails Autoscale keeps your app healthy. Simple and effective autoscaling for Web, Sidekiq, Delayed Job, and Que.

Rails Autoscale sponsor

Tomo 1.0: A Friendly CLI for Deploying Rails Apps — There’s a short tutorial for deploying Rails, and the documentation is thorough.

Matt Brictson

ActiveLdap 6.0: An Object Oriented Interface to LDAP — A very long standing project (16 years!) that has just had an update. LDAP stands for Lightweight Directory Access Protocol and while I don’t hear about it much anymore, it has plenty of established use cases.

Sutou Kouhei

Elasticsearch Integrations for ActiveModel/Record and Rails

Elastic

RubyMine 2020.1 Released

Natalie Kudanova




science and technology

Stimulus Reflex, and sending thanks to Matz

#498 — April 23, 2020

Read on the Web

Ruby Weekly

Credit: Divina Epiphania / Shutterstock.com

Mining for Malicious Ruby Gems: 700+ Gems Affected — Breathe easy as this was all resolved a month ago (and was too obscure to pay off for the hackers anyway) but a security research team recently found over 700 malicious Ruby gems that were subtle typos/adjustments of more popular gems (e.g. atlas-client vs atlas_client – could you tell which one is real?)

Tomislav Maljic

You Can Now Sponsor Matz on GitHub — I appreciate these are challenging times, but if you’ve ever wanted to give a big thank you to Matz, the creator of Ruby, here’s one way to do it. We’re sponsoring Matz now as without him, this newsletter wouldn’t exist! ???? Alternatively, if you have little to spare, maybe send him a thanks on Twitter?

GitHub Sponsors

Ruby Performance Tips — Here’s a collection of practical tips for improving Ruby performance for better user experiences, brought to you by Raygun. Read the tips here.

Raygun sponsor

Full Text Search in Milliseconds with Rails and Postgres — If you’ve never played with full text search with Postgres and Rails, this is a fine place to start. It covers LIKE/ILIKE, trigrams, and ‘proper’ full text searching. We also get to see how Leigh took a query from taking 130ms down to 7ms.

Leigh Halliday

▶  Introduction to Stimulus Reflex — Stimiulus Reflex makes SPA-type interactions very simple by using ActionCable to render pages and then diffing them on the client.

GoRails

Rails Performance: When is Caching the Right Choice? — Before you say “always”, understand that caching is not free and, if done incorrectly, can even make things worse.

Jonathan Miles

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Ruby Backend Developer (Austria) — We’re seeking mid-level and senior devs to join us and build top-class backend infrastructure for our adidas apps, used by millions. Our stack includes: jRuby, Sinatra, Sidekiq, MySQL, & MongoDB.

Runtastic

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

How to Customize Webpack in Rails Apps — How to go about configuring webpack when tweaking webpacker.yml just isn’t enough.

Ross Kaffenberger

RSpec Given/When/Then with Symbols — An interesting, alternative way to structure a BDD feature in RSpec. I think I prefer the underscores but YMMV.

Caius Durling

Looking Inside a Ruby Gem — Piotr decomposes a .gem file which turns out to just be a collection of gzipped and tarred files, only some of which are the code.

Piotr Murach

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, you will learn about the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Passing Rails Controller Params to SidekiqActionController::Parameters can give Sidekiq issues.

Prathamesh Sonpatki

Catchup Subscriptions with Rails Event Store

Miroslaw Praglowski

Logic-less Ruby Templates with Mustache

David Santangelo

▶  Discussing Ruby for Good with Sean Marcia — Sean talks about founding Ruby For Good (an event about philanthropic Ruby development) and some of the projects it has been responsible for creating.

Ruby Rogues podcast

???? Code and Tools

Impressionist 2.0: A Plugin to Log Impressions in Rails Apps — Impressionist tracks page views and impressions. v2.0 has just dropped but they’re also are looking for new maintainers, so contact them if you want to get involved.

Charlotte Ruby Group

acli 0.3: A Command Line Client for Action Cable — Interesting on two fronts.. first, because it’s an mruby app, and we don’t see many of those, and second, because it lets you play with Action Cable channels in any easier way.

Vladimir Dementyev

Undercover: A Tool to Stop You Shipping Untested Code — It’s like RuboCop but for code coverage rather than code style.

Jan Grodowski

How to Monitor Your Host Metrics Automatically

AppSignal sponsor

Bridgetown: A Modern Ruby (JAMstack) Web Framework — Bridgetown is a new Ruby-based static-site generator based on a fork of Jekyll. It supports plugins and Webpack, so you can use your front-end framework of choice.

Bridgetown

net-ssh 6.0: A Pure Ruby Implementation of the SSH2 Client Protocol — Yes, you can write programs that invoke and interact with processes on remote servers, via SSH2, all in Ruby.

Buck, Fazekas, et al.




science and technology

An interview with Ruby ETL expert Thibaut Barrère

#499 — April 30, 2020

Read on the Web

???? Occasionally we run interviews in Ruby Weekly and we're back with another one.. with long time Rubyist and Kiba maintainer, Thibaut Barrère. Be sure to check out the bottom of this issue to read it, especially if you ever run ETL jobs with Ruby! ????

Ruby Weekly

▶  Let's Build a Twitter Clone in 10 Minutes with Rails, CableReady, and StimulusReflex — You know that cloning Twitter in 10 minutes is impossible, but what about the core mechanism of the idea? After a slow first minute, this video does a pretty good job of showing off some techniques you might not have used before.

Nate Hopkins

discuss.rubyonrails: The Rails Project Discussion Forum — Basically a Web version of the Rails mailing lists and a worthwhile place to head if you want to suggest features, ask questions, etc.

Ruby on Rails Discussions

Easy Rails Deployments — Deploy your Ruby, Rails, Sinatra, and Rack application to any cloud or server. Cloud 66 offers a scalable Heroku alternative that doesn't lock you in. Try it free and get extra $66 free credits with the code: 'Ruby-Weekly'.

Cloud 66 sponsor

sequel-activerecord-adapter: Allows Sequel to Reuse an ActiveRecord Connection — If you want to use more Sequel or migrate in that direction, this makes it easy.

Janko Marohnić

Ruby Adds Experimental Support for 'End-Less' Method Definitions — We’ve touched on this in a previous issue, but if you fancy a proper blog post with examples, this is more accessible than digging through feature tracker discussions. I’m not a fan of this syntax myself yet, but Prateek does a good job of selling it.

Prateek Choudhary

CableReady: Trigger Client-Side DOM Changes from Server-Side Ruby — If you skipped the video above because it’s a video (and I know many of you do ????) CableReady is still worth checking out. It aims to “complete the ActionCable story” by providing a way to directly interact with clients over ActionCable WebSockets. The docs will help you get the idea.

Hopsoft

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Security Engineer (Remote) — Are you an engineer with experience in Rails and/or Go? Join our team and help secure our apps and cloud infrastructure.

Shogun

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

A Practical Use for PStorePStore is one of the older parts of Ruby’s stdlib and lets you persist (and restore) a Ruby hash to disk. I haven’t seen it in Ruby code for years but Arkency have found a neat, modern use case.

Paweł Pacana

▶  Drag and Drop Sortable Lists with Rails and Stimulus JS — How to wire up drag and drop lists with a Rails app using Sortable, Stimulus and acts_as_list.

Go Rails

Setting Up Multi-Factor Authentication for RubyGems.org — If you’re a registered user of the official Ruby Gems repository, you should have had an email this week about securing your account using 2FA. This is just my own reminder that this is a good idea especially if you publicly publish widely used gems! :-)

Rubygems.org

Let’s Explore Big-O Notation With Ruby ???? — Learn to use Big-O to look at an algorithm and easily discern its efficiency, without having to run a profiling tool.

Honeybadger sponsor

▶  Discussing Docker and Kubernetes with Kelsey Hightower — A worthwhile show to listen to if Kubernetes and Docker intimidate you but you want to know a little more. Kelsey is good at breaking these things down into understandable pieces.

Rails with Jason Podcast podcast

Rails System Tests in Docker — We’re seeing an uptick in articles about system tests in Rails. Here’s how to integrate them into your development Docker setup.

Hint.io

The Difference Between System Specs and Feature Specs — If you’ve felt the difference between RSpec’s “feature specs” and “system specs” is quite subtle, this explanation will help.

Jason Swett

Why Rubyists Should Consider Learning Go — If you want compilation and a type system, Crystal is probably a better fit for Rubyists, but Go is undoubtedly a neat language and ecosystem (and if you do end up in the Go world, check out our Go weekly! ????)

Ayooluwa Isaiah

Building a Ruby CLI with Thor

Daniel Gómez

???? Code and Tools

git curate: Peruse and Delete git Branches Ergonomically — Got a repo cluttered with branches here and there? git curate aims to cure the pains of getting those branches back under control.

Matt Harvey

MessageBus: A Reliable and Robust Messaging Bus for Ruby and Rack

Sam Saffron

Are You Spending Too Much on Heroku?

Rails Autoscale sponsor

bootstrap_form: A Rails Form Builder for Bootstrap v4-Style Forms

Bootstrap Ruby

ActiveModelAttributes: The Active Record Attributes API, but for Active Model — Brings some of the goodies of the Rails 5 Active Record attributes API to ActiveModel too. 1.6.0 just dropped.

Karol Galanciak

???? A Q&A with…
Thibaut Barrère
Creator of Kiba ETL

Thibaut Barrère is a long-time Rubyist and data engineer who built and maintains the popular Extract, Transform, and Load (ETL) framework Kiba. We asked him some questions about his work:

What inspired you to create Kiba?

A lot of my work since ~2005 has been focused on data integration (making systems speak together), data aggregation etc. I sometimes used GUI-based tools like Microsoft SSIS which, while powerful, are quite far from the coding experience.

I was already using Ruby at that time, and was happy to discover activewarehouse-etl (maintained by Anthony Eden, who runs DNSimple now) providing a Ruby DSL to declare data pipelines. I used it for a while with very good success to implement data extractions and business intelligence ETL, and ultimately took over the maintenance.

In the long run, though (as I explained in a recent Paris.rb talk), the balance between the the cost of OSS maintenance and the usability for my billable and non-billable use-cases proved to be not good enough, which made me decide to stop the maintenance, sadly.

I still wanted to use Ruby to write data pipelines, but I needed to cover more use-cases and reduce the OSS maintenance burden at the same time. This ultimately led me to write and share (in 2015) Kiba ETL, a focused DSL for declarative processing, matching those criterias.

Do you find yourself adding more features while in quarantine?

Before the quarantine, I directed my Kiba bandwith and focus on finalizing Kiba v3 and rewriting the documentation from scratch, to properly encourage best practices I’ve been discovering. I also created experimental branches for Ruby 2.7/2.8 keywords.

During the quarantine, I've reduced client work and OSS work too, to focus on shipping Kiba Pro v2 (which I’ll announce shortly officially). I’ve extracted and generalized (from real-life projects) very useful components, such as a “batch SQL lookup” (useful when replacing relationships keys during data migrations and datawarehouses sync code, in batch rather than row by row), a “file lock” to ensure a single job runs at once, and a “parallel transform” to achieve easy concurrency for things like HTTP queries.

What's the wildest ETL that you've encountered?

Getting the data out of a system which is actively not acting in that direction is always a bit wild.. One can see all types of fancy stuff on the field. For instance, it is not uncommon to have an ETL process start a headless browser, jump through pages, just to get to the CSV/PDF/Excel file that you will then use as your data source!

You can also end up having to figure out ways to read or write very old file formats at times. Recently I wrote a Kiba component to generate a COBOL delimited file, for instance. In large companies, a very widely used I/O is good old SFTP, far away from modern APIs and formats.

Can you tell us how to say your last name? ????

I had to deploy a page to my blog to answer that question properly ????. You’ll find out how to say my name here.

Merci Thibaut!

You can read some of Thibaut's posts on his blog and find out more about Kiba ETL here.




science and technology

A transpiler for futuristic Ruby, and the RailsConf 2020 videos

#500 — May 7, 2020

Read on the Web

???? Welcome to issue 500! A bit of an arbitrary milestone but thanks to you all :-)

Ruby Weekly

Ruby Next: Make All Rubies Quack Alike — Ruby Next is a Ruby-to-Ruby transpiler that allows you to use the latest features of Ruby in previous versions without monkey patching or refinements. Could this be how experimental features are released going forward?

Vladimir Dementyev

Ruby 3 'Guilds' Proposal Now Called Ractor — This documentation is in Japanese (though the source code examples are easy to follow) but the news is that the new, proposed concurrency mechanism for Ruby 3 called Guilds (explained here) has been renamed to Ractor (as in ‘Ruby actors’, Ruby’s take on the actor model).

Koichi Sasada

Don’t Do Auth From Scratch. Focus On Your App — Spend less time on authentication and authorization and more time developing your awesome app. Auth built for <devs>. Download our community edition for free.

FusionAuth sponsor

Take the 2020 Ruby on Rails Survey — This is the sixth outing for Planet Argon’s survey which began in 2009. We try and support it each time as the results always make for interesting reading (see 2018’s results). Participate and become data ????

Planet Argon Team

???? RailsConf 2020 Videos

If you recall, RailsConf 2020 was cancelled in its in-person form to be replaced by a 'couch edition'. This has been taking place and the videos have been released! Here are some of the highlights:

If you want the full collection, here's the YouTube playlist.

Alt::BrightonRuby: A Slightly Odd, Quasi-Conference for Strange Times — Alt::BrightonRuby is not happening in-person this year. Instead, you can buy the recorded talks, get a _why book, and get some podcasts with the speakers.

Alt::BrightonRuby

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Security Engineer (Remote) — Are you an engineer with experience in Rails and/or Go? Join our team and help secure our apps and cloud infrastructure.

Shogun

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

▶  How To Begin Contributing to a Gem — If you’ve been using a library for a while and you want to contribute back, how do you get started? A 12 minute introduction here.

Drifting Ruby

How to Set Up Factory Bot on a Fresh Rails Project — Factory Bot is a library for setting up Ruby objects as test data – an alternative to fixtures, essentially.

Jason Swett

Using Postgres's DISTINCT ON to Avoid an N+1 Query“Recently I fixed a tricky N+1 query and thought I should write it up..”

John Nunemaker

Need to Upgrade Rails? Don’t Know How Long It Will Take? — Get an action plan for your Rails upgrade and an in-depth report about your technical debt and outdated dependencies ????.

FastRuby.io | Rails Upgrade Services sponsor

5 Uses for 'Splats' — 5 different ways to leverage Ruby’s splat (*) operator.

Jason Dinsmore

Running Multiple Instances of Webpacker — If you’re working on multiple Rails apps at once, changing where Rails gets served up is easy by configuring the port, but what about Webpacker? That requires another tweak.

Scott Watermasysk

Performing Asynchronous HTTP requests in Rails — How to update parts an app’s pages with asynchronous HTTP requests. A step-by-step how-to with JavaScript’s fetch() function, and Rails native server-side partial rendering.

Remi Mercier

How to Use AWS SimpleDB from Ruby — If you haven’t heard of AWS SimpleDB, you wouldn’t be alone as it’s not very popular, but it’s a pretty simple and cheap way to store simple documents in the cloud.

Peter Cooper

What's The Difference Between Monitoring Webhooks and Background Jobs

AppSignal sponsor

Ways to Reduce Your Heroku App's Slug Size — You might be surprised Heroku didn’t already do some of this for you.

Rohit Kumar

A Chat with Thibaut Barrère — If you missed our interview with Thibaut Barrere (Rubyist, and creator of the Kiba ETL framework) in last week’s issue, you can catch up here.

Glenn Goodrich

???? Code and Tools

Rodauth 2.0: Ruby's 'Most Advanced' Authentication Framework — A authentication framework that can work in any Rack-based webapp. Built using Roda and Sequel, Rodauth can be used with other frameworks and database libraries if you wish. Why’s it so advanced? More info on that here.

Jeremy Evans

RubyGems 3.1.3 Released — Lots of little bug fixes and tweaks.

RubyGems Blog

Business: Business Day Calculations for Ruby — Define your working days and holidays and then you can do ‘business day arithmetic’ (for example, what’s in 5 working days after now taking holidays and weekends into account?)

GoCardless

Lockbox: Modern Encryption for Rails

Andrew Kane

split: The Rack Based A/B 'Split' Testing Framework — A mature framework with robust configuration and multiple options for determining the winning option.

Split

P.S. In last week's issue, one of the links to our sponsors was incorrect and some readers emailed us to say they really wanted to read the promised article, Let’s Explore Big-O Notation with Ruby, so here it is. Apologies for any inconvenience.




science and technology

npm's CTO: So Long, and Thanks for All The Packages

#334 — April 16, 2020

Read on the Web

Node Weekly

npm Has Now (Actually) Joined GitHub — We announced GitHub’s acquisition of npm a month ago but now the process is complete. Not much real news here but the plan is to now focus on community engagement and improving registry infrastructure.

Jeremy Epling (GitHub)

Node v13.13.0 (Current) Releasedfs.readv is a new function to sequentially read from an array of ArrayBufferViews, util.inspect now lets you specify a maximum length for printed strings, the default maximum HTTP header size has been increased to 16KB, there are three new collaborators, and more.

Michaël Zasso

Get Better Insight into Redis with RedisGreen — Modern hosting and monitoring services include memory usage maps, seamless scaling, key size tracking, and more.

RedisGreen sponsor

▶  Watch the Live Coding of a New Feature for Node.js — This is not something for novices, but if the idea of watching ‘over the shoulder’ of a Node.js collaborator implementing a new feature directly into Node itself interests you.. this could be a valuable hour spent.

Vladimir de Turckheim

node-libcurl 2.1: libcurl Bindings for Nodelibcurl is a very powerful and well established way to fetch data from URLs across numerous protocols. node-libcurl 2.1.0 brings support for the latest version of libcurl (7.69.1) to us in the Node world.

Jonathan Cardoso Machado

npm's CTO: 'So Long, and Thanks for All The Packages!' — Ahmad Nassri was npm’s CTO but has now left. Here, he reflects on the past ten years of npm, the repo, the company, and the achievements of both.

Ahmad Nassri

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Node.js Developer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

▶️ Get ready for your next role: Pluralsight is free for the entire month of April. Stay Home. Skill Up. #FreeApril — SPONSORED

???? Tutorials

Working With AWS Route 53 from Node — Route 53 is Amazon Web Services’ suite of DNS-related services. Like every AWS service, you can control it via an API, and here’s how to manipulate hosted zones from Node.

Valeri Karpov

Best Practices Learnt Running Express.js in Production for 4 Years — There’s a lot of stuff packed in here focused around middleware, testing, logging, and general concerns around scaling and keeping apps running in production.

Adnan Rahić

The Node.js Security Handbook — Improve the security of your Node.js app with the Node.js security handbook made for developers.

Sqreen sponsor

How To Set Up an Express API Backend Project with PostgreSQL — A pretty extensive walkthrough of creating an HTTP API using Express with Node.js and Postgres on the backend, then deploying it all on Heroku.

Chidi Orji

Porting to TypeScript Solved Our API Woes — From the guy behind the (in)famous Wat video comes a tale of porting a backend from Ruby to TypeScript.

Gary Bernhardt

How to Mass Rename Files in Node

Flavio Copes

▶  Let's Build a Digital Circuit Simulator In JavaScript — A special episode of the Low Level JavaScript series takes us on a brief journey into the world of digital logic.

Low Level JavaScript

The Story of How I Created a Way to Port Windows Apps to Linux — We mentioned ElectronCGI recently as a way to let .NET and Node.js code depend upon each other, but here its creator explains more about the how and why.

Rui Figueiredo

How to Create an Alexa Skill with Node — Implementing a custom ‘skill’ for Amazon Alexa by using Node and AWS Lambda.

Xavier Portilla Edo

???? Tools, Resources and Libraries

Node v10.20.1 (LTS) Released — If you’re still using Node 10, don’t use v10.20.0, use this, due to a bug in the .0 release.

Bethany Nicolle Griggs

emoji-regex: A Regular Expression to Match All Emoji-Only Symbols

Mathias Bynens

ip-num: A Library to Work with ASN, IPv4, and IPv6 Numbers — Happy in both Node and the browser.

dadepo

Optimize Node.js Performance with Distributed Tracing in Datadog

Datadog APM sponsor

verify-json: Verify JSON Using a Lightweight Schema — A lighter weight alternative to something like JSON Schema.

Yusuf Bhabhrawala

middle-manager: A Lightweight 'No BS' Presentation Tool — A bit of humor, really. It turns Markdown into basic presentations but then the magic is it detects your ‘BS’ business language so you can remove it ????

Anders




science and technology

Node 14 has been released

#335 — April 23, 2020

Read on the Web

Node Weekly

Node.js 14 Released — Woo-hoo another major release of Node.js is here. v14 now becomes the current ‘release’ line with it becoming a LTS (Long Term Support) release in October.. so production apps would, ideally, remain on v12 for now. So what’s new..?

  • Diagnostic reports are now a stable feature.
  • It's based on V8 8.1.
  • An experimental Async Local Storage API
  • Improvements to streams.
  • An experimental WebAssembly System Interface (WASI) to support future WebAssembly use cases.
  • Bye bye to the ESM module ‘experimental’ warning (though it still is experimental).

Michael Dawson and Bethany Griggs

Learn Hardcore Functional Programming in JavaScript — Join Brian Lonsdorf and learn how to apply such concepts as pure functions, currying, composition, functors, monads and more.

Frontend Masters sponsor

Puppeteer 3.0: Say Hello to Firefox — Best known as a way to headlessly control Chrome from Node, Puppeteer has recently seen some competition in the form of the cross-browser Playwright recently. But competition can be good and Puppeteer now supports Firefox too.

Mathias Bynens

ZEIT Is Now Vercel — You probably best know ZEIT as the creators and maintainers of the popular Next.js React framework and their ‘Now’ deployment and hosting platform.

Vercel

New OpenSSL Security Release To Require Node Updates? Maybe Not.. — A key security update to OpenSSL raised the possibility of widespread Node releases to incorporate the fixes, but initial suggestions are that Node isn’t affected. Fingers crossed!

Sam Roberts

???? Jobs

Node.js Developer at X-Team (Remote) — Join the most energizing community for developers. Work from anywhere with the world's leading brands.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

ℹ️ If you're interested in running a job listing in this newsletter, there's more info here.

???? Articles & Tutorials

OneTesselAway: Building a Real-Time Public Transit Status Device — A developer wanted to know when the next bus would arrive.. while using lots of cool tech, including Node, the OneBusAway API, and the Tessel 2 IoT platform.

Robert McGuire

What is the toJSON() Function? — If an object has a toJSON function, JSON.stringify() calls toJSON() and serializes the return value from toJSON() instead.

Valeri Karpov

Top GitHub Best Practices for Developers - Expanded Guide — Implementing these best practices could save you time, improve code maintainability, and prevent security risks.

Datree.io sponsor

Refactor Your Node and Express APIs to Serverless with Azure Functions — John Papa points out a Microsoft tutorial that walks through the process of taking a Node API serverless with Azure’s Functions service.

John Papa

Querying SQL Server from Node with async/await

Rob Tomlin

Why I Stopped Using Microservices

Robin Wieruch

???? Tools, Resources and Libraries

lazynpm: A Terminal UI for npm — One of those sort of things you don’t realize you need until you give it a go. There’s a four-minute screencast if you want to see how it works without downloading.

Jesse Duffield

node-sqlite3 4.2: Async, Non-blocking SQLite3 Bindings for Node4.2.0 just came out.

Mapbox

ts-gphoto2-driver: A Node Wrapper for libgphoto2libgphoto2 provides a way to control a variety of digital cameras/DSLRs.

Lenzotti Romain

AppSignal Now Supports Node.js: Roadmap for the Coming Weeks

AppSignal sponsor

Rosetta: A General Purpose Internationalization Library in 292 Bytes — Less than 300 bytes, but does have a few dependencies. Aims to be very simple and is targeted at basic string use cases.

Luke Edwards

nodejs-dns 2.0: The Google Cloud DNS Client for Node

Google

node-osc 5.0: Open Sound Control Protocol LibraryOSC is a protocol used to communicate between media devices.

Myles Borins

ts-node: TypeScript Execution and REPL for Node

TypeStrong

???? And One for Fun..

npm trends: Compare NPM Package Downloads — A site to compare package download counts over time. For example, what about koa vs restify vs fastify?

John Potter




science and technology

Can you build Node add-ons in Rust? Yes.

#336 — April 30, 2020

Read on the Web

Be sure to check out the Tools and Libraries section today as there have been quite a lot of (minor) releases.. from MIDI parsing and JPEG decoding to generating TypeScript types from a Postgres database.. maybe there's something for you ????

Node Weekly

Middy 1.0: A Node Middleware Framework for AWS Lambda — Middy’s aim is to make writing serverless functions (hosted on AWS Lambda) easier by providing a familiar middleware abstraction to Node developers. The example in this post shows off the main benefit.

Luciano Mammino

Rust and Node.js: A Match Made in Heaven? — This is technical stuff but using other languages, such as Rust, for building add-ons for Node is an interesting area.

Anshul Goyal

Faster CI/CD for All Your Software Projects Using Buildkite — See how Shopify scaled from 300 to 1800 engineers while keeping their build times under 5 minutes.

Buildkite sponsor

Editly: Slick, Declarative Command Line Video Editing — I’ve long wondered why there isn’t a good way to “code” video editing at the command line other than wrangling with arcane ffmpeg options. Well.. this uses ffmpeg, but it handles a lot of the wrangling for you.

Mikael Finstad

Node v14.1.0 (Current) ReleasedLast week we featured the release of Node 14.0 and 14.1 is already with us. Principally bug fixes, plus an update to the OpenSSL dependency.

Bethany Nicolle Griggs

???? Jobs

Backend Developer (Skien, Norway) — We are looking for a full-stack dev with a solid track record to help us adapt to tomorrow's security requirements.

OKAY

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

ℹ️ If you're interested in running a job listing in this newsletter, there's more info here.

???? Articles & Tutorials

Four Tools for Web Scraping in Node — A walk through of a few different libraries (for scraping and parsing data directly from websites) to see how they work and how they compare to each other.

Sam Agnew

Six Platforms for Hosting a Node App in 2020 — Of course, you can run a Node app pretty much anywhere there’s a server, but some platforms make it easier than others. These all have free tiers too. Glitch, Now.sh (now Vercel) and Heroku are particular favorites of ours at Cooperpress.

Amit Bendor

Getting Started with NuxtJS — Learn how to create Vue.js-powered server-side rendered apps with NuxtJS including configuring an app and deploying it on Heroku.

Timi Omoyeni

The Node.js Security Handbook — Improve the security of your Node.js app with the Node.js security handbook made for developers.

Sqreen sponsor

A Collection of Challenging TypeScript Exercises“The goal: Let everyone play with many different TypeScript features and get an overview of TypeScript capabilities and principles.”

Marat Dulin

Exploring Node.js Internals — It’s reasonably elementary but Aleem Isiaka explains how the internals of Node.js interact with one another on a simple task such as creating a file.

Smashing Magazine

Creating CommonJS-Based npm Packages via TypeScript

Dr. Axel Rauschmayer

Turning Vue Components Into Reusable npm Packages — Outlines how you can reuse Vue components across your projects by automating your process to bundle, test, document, and publish your components.

Sjoerd de voorhoede

???? Tools, Resources and Libraries

Node v12.16.3 (LTS) Released — OpenSSL gets an update, and warnings are no longer printed for modules that use conditional exports or package name self resolution.

Node.js

pm2 4.4 Released: The Node Production Process Manager — A very mature and widely used process manager that includes a load balancer for keeping Node apps alive forever and to reload them without downtime. v4.4 improves the Node 14 compatibility.

Alexandre Strzelewicz

jpeg-js: A Pure JavaScript JPEG Encoder and Decoder — It admits it’s far slower than native alternatives but if you need a pure JavaScript JPEG encoder/decoder, this is where to go.

Eugene Ware

AppSignal Now Supports Node.js: Roadmap for the Coming Weeks

AppSignal sponsor

node-stream-zip: For Fast Reading of ZIP Archives — Reads chunk by chunk rather than all in one go so it’s memory friendly.

Dimitri Witkowski

JZZ: A MIDI Library for Node and Web Browsers — Send, receive and play MIDI messages from both Node and the browser on Linux, macOS and Windows.

Sema

Vegemite: A Pub/Sub State Manager — Inspired by Immer and Redux, full TypeScript support, and sized at only 623 bytes, which includes one dependency.

Luke Edwards

Kanel: Generate TypeScript Types from Postgres

Kristian Dupont

web-worker: Consistent Web Workers for the Browser and Node — In Node it works as a web-compatible Worker implementation atop worker_threads. In the browser it’s an alias for Worker.

Jason Miller

node-csv-parse: A CSV Parser Implementing the stream.Transform API

Adaltas




science and technology

Node 14.2.0, plus Deno 1.0 is coming

#337 — May 7, 2020

Read on the Web

✍️ With a few of the links today, this is a good time to note we sometimes link to things we disagree with or that are controversial if they are newsworthy or of relevance to our community. Inclusion is not always endorsement but you can read any summaries we write alongside items to get our take on things ????

Node Weekly

Node v14.2.0 (Current) Released — The latest version of Node gains a new experimental way — assert.CallTracker — to track and verify function calls and the amount of times they occur. Also, require('console').Console now supports different group indentations

Node.js

Deno 1.0: What You Need to Know — Two years ago Ryan Dahl, the original creator of Node, gave a popular talk called 10 Things I Regret About Node.js where he revealed Deno, his prototype of how he'd build a better V8-based JavaScript runtime. With 1.0 due next week, Deno is poised to be a particularly exciting release and this article does a good job of cruising through the reasons why.

David Else

Enhance Node.js Performance with Datadog APM — Debug errors and bottlenecks in your code by tracing requests across web servers and services in your environment. Then correlate between distributed request traces, metrics, and logs to troubleshoot issues without switching tools or contexts. Try Datadog APM free.

Datadog APM sponsor

Deno Weekly: Our Newest Newsletter — We really like what we see from Deno (above) so far, so we're launching a new newsletter all about it! ???? Rather than keep mentioning Deno in Node Weekly, we'll be giving it its own space. Even if you're not planning to use Deno, feel free to subscribe for a while, see what happens, then unsubscribe if it doesn't suit you — the next issue will drop on 1.0's release (due next Wednesday).

Cooperpress

Controlling GUIs Automatically with Nut.js — Write Node code that clicks on things, opens apps, types, clicks buttons, etc. Works on Windows, macOS and Linux. Hit the GitHub repo to learn more or check out some examples.

Simon Hofmann

A Practical Guide to Node Buffers — You’ll often encounter Buffer objects for holding binary data in the form of a sequence of bytes during interactions with the operating system, working with files, network transfers, etc.

DigitalOcean

???? Jobs

Node.js Developer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

ℹ️ If you're interested in running a job listing in this newsletter, there's more info here.

???? Articles & Opinions

How to Build a REST Service with Fastify — How to build a basic RESTful service using Fastify, a popular Node Web framework focused on performance/low overheads.

Wisdom Ekpot

▶  How to Use Node.js for Load Testing — A straightforward tour of an approach for hitting a Web site over and over from multiple child processes.

Tom Baranowicz

How to Fix ESLint Errors Upon Save in VS Code — A quick fire tip.

David Walsh

Faster CI/CD for All Your Software Projects Using Buildkite — See how Shopify scaled from 300 to 1800 engineers while keeping their build times under 5 minutes.

Buildkite sponsor

Avoiding Memory Leaks in Node: Best Practices for Performance — Covers very similar ground to another memory leak article we linked recently.

Deepu K Sasidharan

'Some thoughts on the npm acquisition..' — The creator of Hapi and an investor in npm Inc. shared his thoughts on GitHub’s acquisition of npm. I disagree with his conclusion (and his views have also caused concern on Twitter) but it’s nonetheless interesting to get views from behind the curtain.

Eran Hammer

???? Tools, Resources and Libraries

npm 6.14.5 Released — Just a couple of minor bug fixes.

The npm Blog

actions-cli: Monitor Your GitHub Actions in Real Time from the Command Line

Tommaso De Rossi

SQL Template Tag: Tagged Template Strings for Preparing SQL Statements — For use with pg and mysql, for example.

Blake Embrey

webpack-blocks: Configure webpack using Functional Feature Blocks

Andy Wermke

JavaScript Error Tracking with AppSignal v1.3.0 is Here

AppSignal sponsor

FarmHash 3.1: A Node Implementation of Google's High Performance Hash FunctionsFarmHash is a family of non-cryptographic hash functions built by Google mostly for quickly hashing strings.

Lovell Fuller

do-wrapper 4.0: A Node Wrapper for the DigitalOcean v2 API

Matthew Major




science and technology

A CLI podcast player built in Go

#308 — April 17, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

Broccoli: Using Brotli Compression to Embed Static Files in Go — There’s been talk about making static file embedding a standard part of Go, but for now you might find this project interesting. It uses the Brotli compression system to embed a virtual file system of static files in your Go executables as tightly as possible.

Aletheia

How Thanos Would Program in Go — An introduction to the Thanos Go Style Guide built for Thanos, the distributed metrics system project, not the Marvel super-villain, BTW ????

Bartek Płotka

Introducing GoLand 2020.1 — A variety of upgrades for Go Modules support, code-editing features that require little to no interaction from the user, an expanded code completion family, and more! Try free for 30 days.

GoLand sponsor

Understanding Bytes in Go by Building a TCP Protocol — There is a lot more in this long-ish tutorial than just learning about bytes. This is great if, let’s say, you are stuck at home and need a challenge. (Note: If you’ve got deja-vu, we linked this in last week’s brief non-issue.)

Ilija Eftimov

Ebiten 1.11.0 Released: The Go 2D Gamedev Library — Ebiten is one of those genuine gems of a project. Maybe use it to take part in this weekend’s Ludum Dare game jam? More Go entries would be neat..

Ebiten

Generics in Go: How They Work and How to Play With Them — Generics are a lot closer than you might think. So much so that you can try them today in a browser or compile locally.

Chris Brown

???? Jobs

Senior Software Engineer (Go) – 100% Remote (UK/EU Only) — Form3 is building the most exciting banking technology on the planet and are looking for Talented Engineers to join the team.

Form3

Golang Developer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

Statically Compiling Go Programs — If you thought all/most Go binaries were static, you might be surprised to find out that some core packages use cgo code and result in dynamically linked libraries.

Martin Tournoij

How To Create Testable Go Code — Structure your code and tests to be mockable, testable, and maintainable, even if it calls external services.

Dave Wales

The Go Security Checklist — Ensure the infrastructure and the code of your Go applications are secure with the latest actionable best practices.

Sqreen sponsor

Build Your Own Neural Network in Go — A beginner’s guide to building the simplest parts of a neural network completely from scratch.

Dasaradh S K

'How I Built a Cloud Gaming System with WebRTC and Go'

Thanh Nguyen

???? Code & Tools

podcast-cli: A Podcast Player with a Terminal-Based Interface

Goulin

Godocgen: A Go Documentation Generator — Godocgen can output to multiple formats/destinations, making it easy to host as a static site. More background here.

Holloway Chew Kean Ho

3mux: An i3-inspired Terminal Multiplexer — Imagine something like tmux but easier to learn and with sensible defaults. Plus, it’s written in Go so you can tweak it as much as you like :-)

Aaron Janse

Micro 2.5: A Go Micro Services Development Framework

Micro

Beta Launch: Code Performance Profiling - Find & Fix Bottlenecks

Blackfire sponsor

Goph: A Native Go SSH Client — Supports connections using passwords, private keys, keys with passphrases, doing file uploads and downloads, etc.

Mohamed El Bahja

GeoDB: A Persistent Geospatial Database with Geofencing and Google Maps Support — Built using Badger gRPC and the Google Maps API. Track the geolocation of objects across boundaries or in relation to other objects.

Coleman Word

oneinfra: A 'Kubernetes as a Service' Platform — Provide or consume Kubernetes clusters at scale, on any platform or service provider.

oneinfra

Gocorona: Track COVID-19 Statistics From Your Terminal — A short and sweet demonstration of what you can throw together quickly using termui, a customizable Go-powered terminal dashboard and widget library.

Ayooluwa Isaiah




science and technology

The 2019 Go developer survey results are available

#309 — April 24, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

Go Developer Survey 2019 Results — The annual survey results are here but calculated differently than in previous years. See how the community feels, what tools we use, and what we’re really using Go for.

The Go Blog

Fiber: An Express.js Inspired Web Framework for Go — If you know Express (from the Node world) than Fiber will look very familiar. It supports middleware, WebSockets, and various template engines, all while boasting a low memory footprint. Built on top of FastHTTP.

Fiber

We Now Offer Remote Go, Docker or Kubernetes Training — We offer live-streaming remote training as well as video training for engineers and companies that want to learn Go, Docker and/or Kubernetes. Having trained over 5,000 engineers, we have carefully crafted these classes for students to get as much value as possible.

Ardan Labs sponsor

A Comparison of Three Programming Languages for Bioinformatics — This is quite an academic piece but basically Go, Java and C++ were put head to head in an intensive bioinformatics task. The good news? Go won on memory usage and beat the C++17 approach (which was admittedly less than ideal) in performance. The team in question chose Go going forward.

BMC Bioinformatics

Go for Cloud — A Few Reflections for FaaS with AWS Lambda — A response to a this article about Go’s pros and cons in the cloud. You should read both.

Filip Lubniewski

???? Jobs

Enjoy Building Scalable Infrastructure in Go? Stream Is Hiring — Like coding in Go? We do too. Stream is hiring in Amsterdam. Apply now.

Stream

Golang Developer at X-Team (Remote) — Join the most energizing community for developers. Work from anywhere with the world's leading brands.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

An Introduction to Debugging with Delve — If you’re in the “I don’t really use a debugger..” camp, Paschalis’s story and brief tutorial might help you dip a toe into the water.

Paschalis Tsilias

Object Ordering in Go — This is all about object comparison and the types of comparisons that are allowed in Go. Reading this post > Not reading this post.

Eyal Posener

How to Manage Database Timeouts and Cancellations in Go — How to cancel database queries from your app and what quirks and edge cases you need to be aware of.

Alex Edwards

The Go Security Checklist — From code to infrastructure, learn how to improve the security of your Go applications with the Go security checklist.

Sqreen sponsor

Data Logging with Go: How to Store Customer Details Securely — Specifically, this looks at using custom protobuf FieldOptions to mark fields as OK to log and reflection to check those options.

Vadzim Zapolski-Dounar

How to Install Go in FreeBSD in 5 Minutes — You can use a package manager, but this way has advantages and it’s easy.

Jeremy Morgan

???? Code & Tools

Fynedesk: A Fyne-Powered Full Desktop Environment for Linux/Unix — Previously we’ve linked to Fyne, a Go-based cross-platform GUI framework, but now it’s been used to create an entire Linux desktop environment!

Fyne.io

Lockgate: A Cross-Platform Locking Library — Has support for distributed locks using Kubernetes and OS file locks support.

Flant

Pomerium: An Identity-Aware Secure Access Proxy — An identity aware access-proxy modeled after Google’s BeyondCorp. Think VPN access benefits but without the VPN. Built in Go, naturally.

Pomerium

Beta Launch: Code Performance Profiling - Find & Fix Bottlenecks

Blackfire sponsor

Apex Log: A Structured Logging Package for Go — Inspired by Logrus.

Apex

mediary: Add Interceptors to the Go HTTP Client — This opens up a few options: tracing, request dumping, statistics collection, etc.

Here Mobility SDK

iso9660: A Go Library for Reading and Creating ISO9660 Images — The use cases for this will be a bit niche. The author created it to dynamically generate ISOs to be mounted in vSphere VMs.

Kamil Domański

pxy: A Go Livestream Proxy from WebSockets to External RTMP Endpoints

Chua Bing Quan




science and technology

An insightful interview with Go's Rob Pike

#310 — May 1, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

An Interview with Go's Rob Pike — Go’s co-creator answers some big picture questions about Go’s status, history, and future. “Go has indeed become the language of cloud infrastructure,” says Rob.

Evrone

???? What's Coming in Go 1.15 — This presentation covers all the major sections: tooling, performance, API changes, and the Big Ones, like the aforementioned smaller binaries. Fingers crossed for a final release in August.

Daniel Martí slidedeck

Troubleshoot Golang App Issues with End-To-End Distributed Tracing — Trace requests across service boundaries to optimize bottlenecks by drilling into individual traces end-to-end with flame graphs. Correlate Golang traces with related logs and metrics for fast troubleshooting. Enhance performance with a free Datadog APM trial.

Datadog APM sponsor

Debugging Go Programs using Delve — The recent Go community survey showed that most Go developers use text-based logging (e.g. with fmt.Print()) to debug, but if you want to step things up a notch, this is a gentle intro to Delve.

Naveen Ramanathan

My Journey Optimizing The Go Compiler — Assel explains how a simple task evolved into a legitimate compiler optimization (aimed at 1.15) and proves we should all have a curious mind.

Assel Meher

The 'Ultimate' Go Study Guide — A large repository of code examples with comments and notes from Hoanh’s attempt at learning the language. If you pick up concepts well from straightforward examples, this is worth a look.

Hoanh An

???? Jobs

Software Engineer at HiPeople (Remote/Berlin) — Fast-moving startup (backed by top tier VCs) shaping the future of modern recruiting is looking for engineers who love working with Go.

HiPeople

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

Making a Multiplayer Game with Go and gRPC — Started as a (somewhat ambitious) project to learn Go, Sam walks us through the algorithms, design decisions, mistakes, and where Go helped and hurt the game.

Samuel Mortenson

Documenting a Go GitHub Repo — Or, “How to Keep the README in Your GitHub Repo in Sync with Your Go Doc.”

Eyal Posener

The 5 Crucial PDF & Office Features For Corporate Apps in Pure Go — UniDoc develops pure Go libraries for managing PDF and Office files since 2016. Here are the features developers use the most.

UniDoc sponsor

▶  Discussing Building Immediate Mode GUIs in Go — Elias Naur, creator of Gio, joins the popular Go podcast to discuss building GUI apps with Go, the pros and cons of immediate vs retained mode and examples of each.

Go Time Podcast podcast

The Creation of a Realtime Patient Monitoring System with Go and Vue in 3 Days — This is the Go content I am here for. Connecting with monitoring devices and leveraging Go’s strengths to create a helpful, distributed application. Great work.

Kasun Vithanage

Add It Up: Azure’s Go Problem — Here’s one takeaway from the Go Developer Survey. Of the major clouds, Azure is the one Go developers seem least enamored by.

Lawrence E Hecht

Why You Should Generally Be using the Latest Version of Go — No surprising arguments here.

Chris Siebenmann

???? Code & Tools

XLSX: A Library for Reading and Writing XLSX (Excel) Files — Got spreadsheets? Want to make spreadsheets? There’s a lot you can do with them here.

Geoffrey J. Teale

SQLBoiler: Generate a Go ORM Tailored to Your Database Schema — A long standing library that has now switched to modules.

Volatile Technologies Inc.

Decimal: Arbitrary-Precision Fixed-Point Decimal Numbers for Go — The library laments that it can only support decimal numbers with up to 2^38 digits after the decimal point so take care ????

Spring Engineering

Beta Launch: Code Performance Profiling - Find & Fix Bottlenecks

Blackfire sponsor

Redigo: A Go Client for Redis — In related news, Redis 6.0 has just been released.

Gary Burd

ntp: Facebook's NTP Libraries — NTP stands for “Network Time Protocol”, if you were wondering. Basically, clock synchronization.

Facebook Incubator

grobotstxt: A Native Go Port of Google's Robots.txt Parser and Matcher Library — Now you can crawl your own site, just like Google does.

Jim Smart

A Compiler for a Small Custom Language Into x86-64 Assembly — One of those ‘labor of love’ type projects that you might enjoy poking around in. You won’t use this project directly, but you might be intrigued how to create a similar compiler for your own thing.

Maurice Tollmien

MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams — A Go reimplementation of this C++ version.

Steve Tan

Liftbridge 1.0: Lightweight, Fault-Tolerant Message Streams — A server that implements a durable, replicated message log for the NATS messaging system.

Liftbridge




science and technology

Caddy 2.0 released, plus a little black hat Go

#311 — May 8, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

Caddy 2: The Go-Powered Web Server with Automatic TLS — After over a year of redesign, Caddy 2 has a new architecture to v1. If you want a new HTTPS server that ‘just works’, Caddy is well worth a look IMO. Its lead creator, Matt Holt, answered lots of questions on this Hacker News thread about the release.

Caddy Web Server

Rek: An Easy HTTP Client for Go — The inspiration here is from Python’s very well known and highly esteemed Requests library.. so the Pythonistas among you might like this!

Luc Perkins

Modern Redis Features with RedisGreen — Online upgrades to the latest Redis 6.0 features, memory mapping, key size tracking, and more.

RedisGreen sponsor

Life Without Line Numbers — There’s a lot of buzz around reducing the size of Go binaries (1.15 does so by ~6%) and here’s another tactic: reduce the precision of the position information. The gain is 2-6%, depending on how far you take it.

Josh Bleecher Snyder

▶  Discussing Black Hat Go“Are you excited to learn about hacking and that?” Got an hour? Roberto Clapis, a security engineer at Google, and Tom Steele, a co-author of Black Hat Go, join the Go Time team to discuss security, penetration testing, and more.

Go Time Podcast

???? Jobs

Enjoy Building Scalable Infrastructure in Go? Stream Is Hiring — Like coding in Go? We do too. Stream is hiring in Amsterdam. Apply now.

Stream

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

Mid-Stack Inlining in Go — Inlining a function can lead to serious performance gains, so why not do it for everything? Well, there are always trade-offs.

Dave Cheney

Asynchronous Preemption in Go 1.14 — How the new preemption implementation works, including the use of a lesser-known signal (SIGURG).

Vincent Blanchon

Why Are My Go Executable Files Larger Than My Source Code? — We built a data visualization tool to find out. Here’s how we built it, and what we learned.

Cockroach Labs sponsor

Accelerating Aggregate MD5 Hashing Up to 800% with AVX512 — The culmination of this work is md5-simd, a Go library that performs such rapid MD5 hashing (when running concurrently). The use cases here are quite restricted but you may appreciate seeing how such things are implemented for any high end SIMD wrangling you need to do one day.

MinIO Blog

▶  A Beginner's Guide to gRPC in Go — There’s a written version of the tutorial if you dislike videos.

TutorialEdge

Four Steps to Daemonize Your Go Programs — Daemons are programs that run as non-interactive background processes (e.g. background job processors, Web servers, database systems).

Ilija Eftimov

Go as a Scripting Language? — There’s plenty of folks that use Go as a scripting language, but there are challenges around REPLs and shebang support. Some of these challenges are being addressed today.

Segio De Simone

???? Code & Tools

UUID 3.3: A Pure Go Implementation of UUIDs — A pure Go implementation of Universally Unique Identifiers (UUID) as defined in RFC-4122 covering versions 1 through 5.

The Go Commune

Reed-Solomon: A Reed-Solomon Erasure Coding Library — A Go port of a Java library built by Backblaze that does Reed Solomon erasure coding (a way to send or store data in a larger form that’s resilient to data loss). Boasts operation of over 1GB/sec per core.

Klaus Post

ko 0.5: Build and Deploy Go Apps on Kubernetes — ko’s objective is to “to make containers invisible infrastructure.” It’s been rapidly maturing in the past few months too.

Google

Monitor the Health and Performance of Your Golang Apps with Datadog APM. Free Trial

Datadog APM sponsor

Tengo 2.2: A Fast Embeddable Script Language for Go — Quite a mature project now and worth a look if you need to add some dynamic scripting to your code.

Daniel Kang

UniPDF 3.7: A Library for Creating and Processing PDF Files — Pure Go, which is neat, but note it’s dual licensed: AGPL for open source, commercial for closed source projects.

UniDoc

Mockery: A Mock Code Generator for Go Interfaces

Vektra

Dynamo: An Expressive DynamoDB Library

Greg Greg

???? Two Fun Side Projects

gasm: An Experimental WASM Virtual Machine for Gophers“I did this just for fun and for learning WASM specification.” Nonetheless, it works with basic examples.

Takeshi Yoneda

thdwb: A Homebrew Web Browser and Rendering Engine — Another experimental, fun learning project. You won’t be using it for your day to day browsing any time soon but projects like this keep the imagination fueled up.

Danilo Fragoso

It'd be quite cool to link to more fun Go experiments and side projects actually, so let us know if you work on any. Bonus points for games, musical, or Web experiences ????




science and technology

React Router & Webpack in Production

I’ve been working on a pretty large react-router codebase at work. Currently it has around 50~ code splits, which as you can imagine, is a lot of routes. This is going to be a post on the things I’ve learned throughout building out my development / production config and how we are using webpack in production.

###Initial Setup

Before I really dive into how my webpack config is setup and the problems I’ve found, I’ll quickly go over how this app is setup. Currently, there’s one entry point and it looks like this:

import React from 'react'
import { render } from 'react-dom'
import { match, Router, browserHistory } from 'react-router'
import AsyncProps from 'async-props'
import routes from '../routes/index'
/* globals document, window */

const { pathname, search, hash } = window.location
const location = `${pathname}${search}${hash}`

match({ routes, location }, () => {
  render(
    <Router
      render={props => <AsyncProps {...props}/>}
      routes={routes}
      history={browserHistory}
    />,
    document.getElementById('app')
  )
})

It looks like a standard react-router setup, except a couple things are different. For one, there’s way too many routes to have them all in this file, so we are importing the main route object into this file. Second, we are using match on the client side. Without matching first, the client side would try to render before the splits were downloaded causing an error. You can read a little more about match on the client here.

Next, we are using Ryan Florence’s awesome async-props library for loading data into components. It allows me to load data from an api before the server renders components. It will pass the data down to the client for the client-side render, and then data will load as you navigate to new pages automatically.

###Routes

Our main routes file looks like this:


export default {
  component: 'div',
  path: '/',
  indexRoute: require('./index'),
  childRoutes: [
    require('./login'),
    require('./account'),
    ...
  ]
}

There’s a lot more require’s in our app of course. And these are nested pretty deep. The files referenced in the root file have more child routes, and those use require.ensure which you can read about in the webpack docs on code splitting. It tells webpack to make a new bundle, and then load that bundle when require.ensure is called on the client. Here’s an example:

if(typeof require.ensure !== "function") require.ensure = function(d, c) { c(require) }

module.exports = {
  path: 'account',
  getComponent(location, cb) {
    require.ensure([], (require) => {
      cb(null, require('../../views/master/index.jsx'))
    })
  },
  childRoutes: [
    require('./settings'),
  ]
}

There’s a few things going on here. First, we have a function at the top that will polyfill require.ensure. Why? Well, on this project we are server rendering our whole site as well, which I would rather not do, but due to the type of site we are building: we have to. The next thing is the relative require path. I’m using this awesome babel resolver plugin along with webpack’s resolve paths so that I can import files like this:

import Header from '../../master/header'
//becomes
import Header from 'master/header'

Why do I have to use a babel plugin AND webpack’s resolve feature? Once again, doing a server rendered app, the code is ran on the server and also through webpack. In this particular app, I haven’t had time to experiment with webpacking the server. Anyways, if I didn’t use the babel plugin, errors would be thrown on the server, but webpack would work fine. This is one of the common things I have ran into while building this app.

Realizing some things need to be done slightly different on the server or client. You may still be wondering why I am referencing the component as a relative path in the above route example, and that’s because the babel plugin I’m using only works with import and not require. My route objects are the one place that I have these “nasty” looking paths.

##Webpack

I was prompted to make this article after tweeting this out:

A couple people wanted a better explanation as to what’s happening here. When I was first building my production webpack config, even after using all of these plugins:

new webpack.optimize.CommonsChunkPlugin('vendor', 'vendor.js'),
new webpack.optimize.OccurenceOrderPlugin(),
new webpack.optimize.DedupePlugin(),
new webpack.optimize.UglifyJsPlugin({
  compress: { warnings: false },
  comments: false,
  sourceMap: false,
  mangle: true,
  minimize: true
}),

My bundle looked like this:

That’s pretty huge if you think about it. And I’m not talking about the amount of bundles. I’m talking about the file size. After searching everywhere for a solution to get the bundle size down further, I found webpack’s AggressiveMergingPlugin. This thing is a life saver. As you may have seen from the tweet, the output turns into this:

Just having the main, vendor, and one other bundle brings the whole site under 1MB. I’m using the plugin to only merge files if the size reduction is more than 50%, which is the default.

People talk about code splitting in webpack and think it’s really amazing to load the JS for the page you’re on and nothing more. It sounds great. The problem is that the file size is immensely bigger. If someone more familiar with webpack has a better idea as to why this is, I’d like a better explanation. It isn’t feasable to keep the splits instead of merging them. This site is pretty large, with a lot of routes as you can tell from the screenshots. Codesplitting without merging would cause way more waiting on the client side every time you navigate to a new page. Even if the JS was heavily cached, the first time you hit these pages it will have to load a 300kb bundle for some of them.

##Caching

That takes us to caching. We are about a month away from publicly launching this site, so we haven’t setup the workflow for pushing updates through a cdn, but that will be the end result. For now, in my webpack config, my output object looks like this:

output: {
  path: __dirname + '/public/assets/js/[hash]/',
  filename: '[name].js',
  chunkFilename: '[id].js',
  publicPath: '/assets/js/[hash]/'
},

This is in the production config of course. This way I can cache the files and when I update the code, the hash will change and the browser won’t be caching the old code. I pass in the hash as an env variable at runtime to that the server has the correct path to the assets folder.

##Problems

There were a few big problems I came across while building out a server rendered app with dynamic routes. The first was page titles. How am I supposed to have the right title on the client and on the initial server render? Thankfully, Ryan has yet another solution. react-title-component solves this perfectly.

The next was, how do I hit an api, wait for the response on server render, load new data on route changes, and of course, do this at the component level. As I mentioned before, async-props solves this problem too. It will give you route info so that you can make requests based on things in the url.

The next problem is one that I haven’t fully solved. Webpack is getting really slow. It takes around 20 seconds on a maxed out macbook 15” to build code in production. On the server, it takes more like a minute! If I’m in development mode, it takes around 10 seconds to make the initial build, and sometimes it lags on building the splits on code change. If anyone has insight into this I would love to hear it.

This one goes along with the webpack one, and it is reloading the server. I haven’t tried to webpack the server but I hear doing so works great for this. I don’t think it would fix the problem with webpack being slow though, and in fact it would probably make it even slower.

##Folder structure

I almost forgot to throw this one in here! I’m really happy with the structure of this project. I have a views folder that has all of the same folders and file names as the routes folder. It makes it really easy to find things. These also correspond with the URL to the page. /account/settings will be in views/account/settings.jsx and routes/account/settings.js. The same is true for my tests folder.

##Conclusion

I hope this gave you a good glimpse at how webpack and react router work at a larger scale than you see most blog posts cover. If you have any questions or things that you would like me to talk about that I haven’t already, please leave a comment below and I will update this post! I’m sure that I forgot a few problems and tips writing this. I was thinking this would be a short post but it blew up on me!




science and technology

Building Redux Middleware

After writing my post a few months ago on building your own redux app, I have been asked a couple times to write a guide on creating redux middleware and how it works. This will be a quick post on how you can acheive anything with your own middleware!

##Basic middleware


const customMiddleware = store => next => action => {
  if(action.type !== 'custom') return next(action)
  //do stuff!
}

Applying it:

import { createStore, applyMiddleware, } from 'redux'
import reducer from './reducer'
import customMiddleware from './customMiddleware'

const store = createStore(
  reducer,
  applyMiddleware(customMiddleware)
)

Whaaa? store => next => action => I know that looks confusing. Essentially you are building a chain of functions, it will look like this when it gets called:

//next looks something like this:
let dispatched = null
let next = actionAttempt => dispatched = actionAttempt 

const dispatch = customMiddleware(store)(next)

dispatch({
  type: 'custom',
  value: 'test'
})

All you are doing is chaining function calls and passing in the neccesary data. When I first saw this I was confused a little due to the long chain, but it made perfect sense after reading the article on writing redux tests.

So now that we understand how those chained functions work, let’s explain the first line of our middleware.

if(action.type !== 'custom') return next(action)

There should be some way to tell what actions should go through your middleware. In this example, we are saying if the action’s type is not custom call next, which will pass it to any other middleware and then to the reducer.

##Doing Cool stuff

The official guide on redux middleware covers a few examples on this, I’m going to try to explain it in a more simple way.

Say we want an action like this:

dispatch({
  type: 'ajax',
  url: 'http://api.com',
  method: 'POST',
  body: state => ({
    title: state.title
    description: state.description
  }),
  cb: response => console.log('finished!', response)
})

We want this to do a post request, and then call the cb function. It would look something like this:

import fetch from 'isomorphic-fetch'

const ajaxMiddleware = store => next => action => {
  if(action.type !== 'ajax') return next(action)
  
  fetch(action.url, {
    method: action.method,
    body: JSON.stringify(action.body(store.getState()))
  })
  .then(response => response.json())
  .then(json => action.cb(json))
}

It’s pretty simple really. You have access to every method redux offers in middleware. What if we wanted the cb function to have access to dispatching more actions? We could change that last line of the fetch function to this:

.then(json => action.cb(json, store.dispatch))

Now in the callback, we can do:

  cb: (response, dispatch) => dispatch(newAction(response))

As you can see, middleware is very easy to write in redux. You can pass store state back to actions, and so much more. If you need any help or if I didn’t go into detail enough, feel free to leave a comment below!




science and technology

How to Make Your React Apps 15x Faster

Without any modifications, React is really fast as-is. There are, however, a few things that you can do to improve performance. While working at HelloSign, I discovered some quick fixes that made our apps incredibly snappy. With these simple changes, I was able to reduce render time from over 3000 milliseconds to less than 200 milliseconds.

Without any modifications, React is really fast as-is. There are, however, a few things that you can do to improve performance. While working at HelloSign, I discovered some quick fixes that made our apps incredibly snappy. With these simple changes, I was able to reduce render time from over 3000 milliseconds to less than 200 milliseconds.

Editor’s Note:

Check out our upcoming React University Workshops. Our next workshop, React 2016, will be held on April 23 at Microsoft Reactor in San Francisco and will offer a deep dive into creating modern Single-Page Applications (SPA) using React, Redux, React Router, Immutable.js, and Webpack. Also, if you’re interested in learning the basics about what it takes to be a Data Visualization Engineer, check out React and D3.

Introduction

HelloSign is a cloud-based electronic signature tool founded in 2010. As you can imagine, HelloSign is a very JavaScript-heavy codebase. A lot of client-side behavior is necessary to create a rich signing experience. Lately, we’ve moved much of our codebase toward React. In fact, in many places we’ve broken up our codebase into several single-page applications written in React.

Although the HelloSign team was happy with React’s performance before I initially joined the project, I quickly found some low-hanging fruit that could improve runtime speed. Here are the steps you should take to see similar improvements in your own applications.

Create a Baseline Performance Measurement

Before you begin, you should take a baseline measurement. Optimizations are meaningless if you can’t verify the results of your modifications.

Thankfully, Chrome has excellent developer tools to help. One, little-used feature of Chrome’s DevTools is the “Timeline” tool. It allows you to record and analyze all activity in your application. You can record interactions on the page, locate potential memory leaks, measure the total time it takes to perform a task, and identify areas of potential jank. Best of all, the results can be recorded for comparison with your final benchmark.

There’s actually a really awesome video on Chrome’s DevTools that goes into detail about the “Timeline” feature. You can view it here.

We chose to measure the time elapsed between the initial paint of our signer page to the final rendering of the entire page. The initial download of our bundles still needs some optimization, but we’re neither going to mess with nor measure this parameter. It’s fairly easy and consistent to test render time rather than trying to click areas around the page and trying to measure its performance in a repeatable way. Then, all we needed to do was to go to the signer page, open Chrome’s DevTools “Timeline” tab, and refresh the page.

As a side note, make sure that when performing this test, the “Paint” and “Screenshots” boxes are checked so that you can see what the user sees as the page is being rendered.

After all that, we determined that our rendering time from initial paint was a little over 3 seconds. Much too long. Luckily, there was little we had to do to make this quite a bit faster.

Set NODE_ENV to Production

This step is easy to get wrong, even if you are well-informed. React’s documentation provides an overview, but doesn’t provide many specifics. React has great developer warnings and error checking, but these are only intended for development; if you take a look at React’s source code, you’ll see a lot of if (process.env.NODE_ENV != 'production') checks. This is running extra code that is not needed by the end user, not to mention that calling process.env.NODE_ENV is extremely slow. For production environments, we can remove all this unnecessary code. Just keep in mind that you don’t want to do this in development because it will remove all those helpful developer warnings.

If you’re using Webpack, you can use DefinePlugin to replace all instances of process.env.NODE_ENV with 'production', and then use the UglifyJsPlugin to remove all the dead code that no longer runs. Here’s a sample setup that you might use:

// webpack.config.js
  ...
  plugins: [
    new webpack.DefinePlugin({
      // A common mistake is not stringifying the "production" string.
      'process.env.NODE_ENV': JSON.stringify('production')
    }),
    new webpack.optimize.UglifyJsPlugin({
      compress: {
        warnings: false
      }
    })
  ]
  ...

React Constant and Inline Elements Transforms

React 0.14 introduced support for certain transpile time optimizations with Constant and Inline Element Babel Transforms. React Constant Elements treats JSX elements as values and hoists them to a higher scope. In other words, it hoists static elements and thereby reduces calls to React.createClass. React Inline Elements converts JSX elements into the object literals that they eventually return. Again, this minimizes the runtime calls to React.createClass.

The implementation is rather simple. We added our Babel configuration in our package.json file:

// package.json
  ...
  "babel": {
    "env": {
      "production": {
        "plugins": [
          "transform-react-constant-elements",
          "transform-react-inline-elements"
        ]
      }
    }
  },
  ...

Final Measurement / Conclusion

Lastly, you’ll want to run the benchmark again and compare it with that saved benchmark from before these optimizations. As you can see, the total runtime profile ends 200ms after initial paint! That’s 15 times faster!




science and technology

Playing With React and D3

D3 is great at data visualizations, but it manipulates the DOM directly to display that data. Rendering DOM elements is where React shines. It uses a virtual representation of the DOM (virtual DOM) and a super performant diffing algorithm in order to determine the fastest way to update the DOM. We want to leverage React’s highly efficient, declarative, and reusable components with D3’s data utility functions.

At this point, we can safely say that React is the preferred JavaScript library for building user interfaces. It is used practically everywhere and is almost as pervasive as jQuery. It has an API that is simple, powerful, and easy to learn. Its performance metrics are really impressive thanks to the Virtual DOM and its clever diff algorithm between state changes. Nothing, however, is perfect, and React too has its limitations. One of React’s greatest strengths is the ease with which it integrate third-party libraries, but some libraries, especially opinionated ones, are more difficult to integrate than others.

An extremely popular library that can be tricky to integrate with React is D3.js. D3 is an excellent data visualization library with a rich and powerful API. It is the gold standard of data visualizations. However, Because this library is opinionated about data, it is no trivial endeavour to get it to work with React. A few simple strategies permit these two libraries to work together in very powerful ways.

Editor’s Note: Check out our upcoming workshop, React and D3, a crash course in learning how to create data visualizations with these two in demand libraries. Reserve your spot now on Eventbrite and get 20% off admission. Learn more at the Eventbrite page

What is React?

React is an open-source JavaScript library for creating user interfaces that addresses the challenges of building large applications with data that changes over time. Originally developed at Facebook, it is now seen in many of the most commonly used web applications including Instagram, Netflix, Airbnb, and HelloSign.

React helps developers build applications by helping manage the application state. It’s simple, declarative, and composable. React is not a traditional MVC framework because React is really only interested in building user interfaces. Some have called it the “V(iew)” in MVC, but that’s a little misleading. React’s viewpoint is different. As application logic has reoriented toward the client, developers have applied more structure to their front-end JavaScript. We applied a paradigm that we already understood from the server (MVC) to the browser. Of course, the browser environment is very different from the server. React acknowledges that client-side applications are really a collection of UI components that should react to events like user interaction.

React encourages the building applications out of self-contained, reusable components that only care about a small piece of the UI. Other frameworks such as Angular also attempt to do this, but React stands out because it enforces a unidirectional data flow from parent component to child component. This makes debugging much easier. Debugging is the hardest part of application development, so while React is more verbose that other libraries or frameworks, in the end it saves a lot of time. In a framework like Angular’s, it can be hard to figure out where a bug is coming from: The view? The model? The controller? The directive? The directive controller? Data in Angular flows in many different directions, and this makes it hard to reason about that state of your application. In React, when there is a bug (and there will be!), you can quickly determine where the bug originated from because data only moves in one direction. Locating a bug is as simple as connecting the numbered dots until you find the culprit.

What is D3?

D3 (Data-Driven Documents) is a JavaScript library for producing dynamic, interactive data-visualizations. It’s fairly low level, and the developer has a lot of control over the end result. It takes a bit of work to get D3 to do what you want, so if you’re looking for a more prepackaged solution, you’re probably better off with highcharts.js. That said, it is fairly simple to pick up once you get the hang of it.

D3 does four main things:

  1. LOADS: D3 has convenient methods for importing data from CSV documents.
  2. BINDS: D3 binds data elements to the DOM via JavaScript and SVG.
  3. TRANSFORMS: data can be adjusted to fit your visual requirements
  4. TRANSITIONS: D3 can respond to user input and animate elements based on that input

Why Would We Want To Use React with D3?

D3 is great at data visualizations, but it manipulates the DOM directly to display that data. Rendering DOM elements is where React shines. It uses a virtual representation of the DOM (virtual DOM) and a super performant diffing algorithm in order to determine the fastest way to update the DOM. We want to leverage React’s highly efficient, declarative, and reusable components with D3’s data utility functions. Also, once we create a chart component, we can want to be able to reuse that chart with different data anywhere in our app.

How to use React and D3?

D3, like React, is declarative.D3 uses data binding, whereas React uses a unidirectional data flow paradigm. Getting these two libraries to work together takes a bit of work, but the strategy is fairly simple: since SVG lives in the DOM, let React handle displaying SVG representations of the data and lett D3 handle all the math to render the data.

Of course, we’ll have to make compromises. React is unopinionated and flexible, thereby allowing you to accomplish whatever needs to be done. Some tasks, like creating axes, are tedious. We can let D3 directly access the DOM and create. It handles axes well, and since we only need to create very few, this tactic won’t affect performance.

Let’s go through a simple example. I created a repository you can use to follow along here: playing-with-react-and-d3. You can follow in the unfinished directory and if you get stuck you can take a look at the finished directory.

Let’s generate a random list of X-Y coordinates and display them on a ScatterPlot chart. If you’re following the tutorial, a finished example is provided for you under the “finished” directory, but you can also follow along under “unfinished.” I’ve gone through the trouble of doing all the setup for you. The build will automatically be created from “unfinished/src/index.jsx”

Let’s start by creating a simple “Hello World!” React component. Create a file under “components” named “chart.jsx”

// unfinished/src/components/chart.jsx
import React from 'react';

export default (props) => {
  return <h1>Hello, World!</h1>;
}

This example is simple, but let’s go over the explanation anyway. Since we’re rendering a simple H1 with no state, we can just export a function that returns the HTML we expect. If you’re familiar with Angular or Ember, it might look weird to insert HTML directly into our JS code. On the one hand, this goes against everything we’ve learned about unobtrusive JavaScript. But on the other hand, it actually makes sense: we’re not putting JavaScript in our HTML, we’re putting our HTML into our JavaScript. React sees HTML and client-side JavaScript as fundamentally bonded together. They’re both concerned about one thing – rendering UI components to the user. They simply cannot be separated without losing the ability to see what your component is going at a glance. The great benefits of this approach is that you can describe exactly what your component will look like when it’s rendered.

Also, keep in mind that this is only possible with JSX, which translates HTML elements into React functions that will render the HTML to the page.

Now, let’s move on and mount our component to the DOM. Open up “index.jsx”

// unfinished/src/index.jsx
import './main.css';
import React    from 'react';
import ReactDOM from 'react-dom';
import Chart    from './components/chart.jsx';

const mountingPoint = document.createElement('div');
mountingPoint.className = 'react-app';
document.body.appendChild(mountingPoint);
ReactDOM.render(<Chart/>, mountingPoint);

You probably noticed a few things. You might be wondering why we’re requiring a CSS file. We’re using Webpack, which allows us to require CSS files. This is very useful when we modularize both our stylesheets and our JavaScript. We’re also creating a div in which we want to mount our React app. This is just a good practice in case you want to do other things on the page then render a React component. Lastly, we’re calling render on ReactDOM with 2 arguments, the name of the component and the DOM element we want to mount it on.

Now, let’s install all the dependencies by navigating to the unfinished directory and running npm i. Then, fire up the server with npm run start and go to localhost:8080

Awesome! We have rendered our first React component! Let’s do something a little less trivial now.

Let’s compose some functions that will create an array of random data points and then render a scatter plot. While we’re at it, we’ll add a button to randomize the dataset and trigger a re-render of our app. Let’s open up our Chart component and add the following:

// unfinished/src/components/chart.jsx
import React       from 'react';
import ScatterPlot from './scatter-plot';

const styles = {
  width   : 500,
  height  : 300,
  padding : 30,
};

// The number of data points for the chart.
const numDataPoints = 50;

// A function that returns a random number from 0 to 1000
const randomNum     = () => Math.floor(Math.random() * 1000);

// A function that creates an array of 50 elements of (x, y) coordinates.
const randomDataSet = () => {
  return Array.apply(null, {length: numDataPoints}).map(() => [randomNum(), randomNum()]);
}

export default class Chart extends React.Component{
  constructor(props) {
    super(props);
    this.state = { data: randomDataSet() };
  }

  randomizeData() {
    this.setState({ data: randomDataSet() });
  }

  render() {
    return <div>
      <h1>Playing With React and D3</h1>
      <ScatterPlot {...this.state} {...styles} />
      <div className="controls">
        <button className="btn randomize" onClick={() => this.randomizeData()}>
          Randomize Data
        </button>
      </div>
    </div>
  }
}

Since we want our component to manage it’s own state, we need to add a bit more code than was necessary for our previous “Hello World” stateless functional component. Instead of just a function, we’re going to extend React.Component and describe our component in the render() method. render() is the heart of any React component. It describes what our component is supposed to looks like. React will call render() on initial mount and on every state change.

Inside of render(), we are both rendering a scatter plot component as if it were an HTML element and setting some properties or “props”. The ... syntax is a convenient JSX and ES2015 spread operator that spreads the attributes of an array or object instead of doing all of that explicitly. For more information check out: JSX Spread Attributes. We’re going to use render() to pass our data and a style object that will be used by some of our child components.

In addition, we’re also rendering a button with an onClick event handler. We’re going to wrap this.randomizeData() with an arrow function and bind the value of this to our Chart component. When the button is clicked, randomizeData() will call this.setState() and pass in some new data.

Let’s talk about this.setState(). If render() is the heart of a React component, setState() is the brains of a component. setState explicitly tells React that we’re changing the state, thereby triggering a re-render of the component and its children. This essentially turns UI components into state machines.

Inside of setState(), we’re passing an object with data set to the randomDataSet(). This means that if we want to retrieve the state of our application, we need only call this.state.whateverStateWereLookingFor. In this case, we can retrieve the randomData by calling this.state.data.

A little side note on how React works: React offers great performance for rendering UI components by implementing a diff algorithm and comparing a virtual DOM in memory with the actual DOM. When you think about it, the DOM is really a large tree structure. If there’s one thing we have learned from decades of computer science research, it’s how to compare and manipulate trees. React takes advantage of clever tree diffing algorithms, but in order to work, each component can only render one parent element (i.e., you cannot render sibling elements). That’s why In the render function we’re wrapping all our elements in one parent div.

Let’s get started with the scatter plot component. Create a file unfinished/src/components/scatter-plot.jsx :

// unfinished/src/components/scatter-plot.jsx
import React        from 'react';
import d3           from 'd3';
import DataCircles  from './data-circles';

// Returns the largest X coordinate from the data set
const xMax   = (data)  => d3.max(data, (d) => d[0]);

// Returns the highest Y coordinate from the data set
const yMax   = (data)  => d3.max(data, (d) => d[1]);

// Returns a function that "scales" X coordinates from the data to fit the chart
const xScale = (props) => {
  return d3.scale.linear()
    .domain([0, xMax(props.data)])
    .range([props.padding, props.width - props.padding * 2]);
};

// Returns a function that "scales" Y coordinates from the data to fit the chart
const yScale = (props) => {
  return d3.scale.linear()
    .domain([0, yMax(props.data)])
    .range([props.height - props.padding, props.padding]);
};

export default (props) => {
  const scales = { xScale: xScale(props), yScale: yScale(props) };
  return <svg width={props.width} height={props.height}>
    <DataCircles {...props} {...scales} />
  </svg>
}

There’s a lot going on here, so let’s start with the stateless functional component that we’re exporting. D3 uses SVG to render data visualizations. D3 has special methods for creating SVG elements and binding data to those elements – but we’re going to let React handle that. We’re creating an SVG element with the properties passed in by the Chart component and which can be accessed via this.props. Then we’re creating a DataCircles component (more on that below) which will render the points for the scatter plot.

Let’s talk about D3 scales. This is where D3 shines. Scales takes care of the messy math involved in converting your data into a format that can be displayed on a chart. If you have a data point value 189281, but your chart is only 200 pixels wide, then D3 scales converts that value to a number you can use.

d3.scale.linear() returns a linear scale. D3 also supports other types of scales (ordinal, logarithmic, square root, etc.), but we won’t be talking about those here. domain is short for an “input domain”, i.e., the range of possible input values. It takes an array of the smallest input value possible and the maximum input value. range on its own is the range of possible output values. So in domain, we’re setting the range of possible data values from our random data, and in range we’re telling D3 the range of our chart. d3.max is a D3 method for determining the maximum value of a dataset. It can take a function which D3 will use to give the max values of the X and Y coordinates.

We use the scales to render the data circles and our axes.

Let’s create the DataCircles component under unfinished/src/components/data-circles.jsx

// unfinished/src/components/data-circles.jsx
import React from 'react';

const renderCircles = (props) => {
  return (coords, index) => {
    const circleProps = {
      cx: props.xScale(coords[0]),
      cy: props.yScale(coords[1]),
      r: 2,
      key: index
    };
    return <circle {...circleProps} />;
  };
};

export default (props) => {
  return <g>{ props.data.map(renderCircles(props)) }</g>
}

In this component, we’re rendering a g element, the SVG equivalent to a div. Since we want to render a point for every set of X-Y coordinates, were must render multiple sibling elements which we wrap together in a g element for React to work. Inside of g, we’re mapping over the data and rendering a circle for each one using renderCircles. renderCircles creates an SVG circle element with a number of properties. Here’s where we’re setting the x and y coordinates (cx and cy respectively) with the D3 scales passed in from the scatter plot component. r is the radius of our circle, and key is something React requires us to do. Since we’re rendering identical sibling components, React’s diffing algorithm needs a way to keep track of them as it updates the DOM over and over. You can use any key you like, as long as it’s unique to the list. Here we’re just going to use the index of each element.

Now, when we look at our browser, we see this:

We can see our random data and randomize that data via user input. Awesome! But we’re missing a way to read this data. What we need are axes. Let’s create them now.

Let’s open up ScatterPlot.jsx and add an XYAxis component

// unfinished/src/components/scatter-plot.jsx

// ...

import XYAxis       from './x-y-axis';

// ...

export default (props) => {
  const scales = { xScale: xScale(props), yScale: yScale(props) };
  return <svg width={props.width} height={props.height}>
    <DataCircles {...props} {...scales} />
    <XYAxis {...props} {...scales} />
  </svg>
}

Now, let’s create the XYAxis component;

// unfinished/src/components/x-y-axis.jsx
import React  from 'react';
import Axis   from './axis';

export default (props) => {
  const xSettings = {
    translate: `translate(0, ${props.height - props.padding})`,
    scale: props.xScale,
    orient: 'bottom'
  };
  const ySettings = {
    translate: `translate(${props.padding}, 0)`,
    scale: props.yScale,
    orient: 'left'
  };
  return <g className="xy-axis">
    <Axis {...xSettings}/>
    <Axis {...ySettings}/>
  </g>
}

For simplicity’s sake, we’re creating two objects which will hold the props for each of our X-Y axes. Let’s create an axis component to explain what these props do. Go ahead and create axis.jsx

// unfinished/src/components/x-y-axis.jsx
import React from 'react';
import d3    from 'd3';

export default class Axis extends React.Component {
  componentDidMount() {
    this.renderAxis();
  }

  componentDidUpdate() {
    this.renderAxis();
  }

  renderAxis() {
    var node  = this.refs.axis;
    var axis = d3.svg.axis().orient(this.props.orient).ticks(5).scale(this.props.scale);
    d3.select(node).call(axis);
  }

  render() {
    return <g className="axis" ref="axis" transform={this.props.translate}></g>
  }
}

Our strategy up to this point has been to let React exclusively handle the DOM. This is a good general rule, but we should leave room for nuance. In this case, the math and work necessary in order to render an axis is quite complicated and D3 has abstracted that pretty nicely. We’re going to let D3 have access to the DOM in this case. And since we’re only going to render a ma




science and technology

Routing in React Native with Jake Murzy

Jake Murzy has been hard at work creating a new navigational library for React Native over the last couple of months. While React JS has the benefit of the highly-regarded React Router, such a comprehensive routing solution doesn’t exist yet in the React Native community. In fact, React Native’s routing landscape has been in constant upheaval for the last year. The library itself has official three ‘navigators’ for handling decision making on which components to show the user, including ‘NavigatorIOS’, ‘Navigator’, and - more recently - ‘NavigatorExperimental’. The open source community likewise has the packages ‘React Native Router Flux’, ‘React Native Router Native’, and ‘React Native Redux Router’, which of which are in various states of completion, or, more commonly, disrepair.

Jake Murzy has been hard at work creating a new navigational library for React Native over the last couple of months. While React JS has the benefit of the highly-regarded React Router, such a comprehensive routing solution doesn’t exist yet in the React Native community. In fact, React Native’s routing landscape has been in constant upheaval for the last year. The library itself has official three ‘navigators’ for handling decision making on which components to show the user, including ‘NavigatorIOS’, ‘Navigator’, and - more recently - ‘NavigatorExperimental’. The open source community likewise has the packages ‘React Native Router Flux’, ‘React Native Router Native’, and ‘React Native Redux Router’, which of which are in various states of completion, or, more commonly, disrepair.

React Router Native appears to focus on matching the API of the immensely popular React Router package, even going as far as introducing the concept of a URL into React Native, which bucks the notion that only web applications need or deserve a URL.


Today Jake is going to share some of his thoughts about his new library.

Q: Hi Jake! The React Native library contains several navigation solutions and the surrounding ecosystem has multiple routing libraries. What made you decide to make your own?

Hey! Thanks for reaching out. I’ve been eagerly watching what’s happening with navigation on React Native for a while. Until very recently, the whole Navigation scene in React Native was a mess. Navigator was being deprecated in favor of NavigationExperimental and NavigationExperimental wasn’t ready for prime time.

My team was just starting a new project so I tried quite a few of the available solutions. Having successfully used React Router on the web, we were looking for a similar solution. Unfortunately, React Router did not support React Native, and other solutions we found were either very unstable, had a hard time keeping up with upstream changes on each release or the quality of code was quite poor.

NavigationExperimental did most of what we wanted but it was a bit too low level so often times we found ourselves writing navigation related code and you can imagine how this gets tedious fast. The low level nature of NavigationExperimental is really by design to allow abstractions to be built up in higher layers. So to finally answer your question, the project came directly out of my frustration trying to make navigation work on React Native as good as React Router did on the web.

Q: What is the strength of your routing system? Is there any type of app that would be a perfect fit with React Router Native? Conversely, is there any type of app that wouldn’t be a good fit with the library?

The use cases for React Router Native is pretty much the same as NavigationExperimental—which is the only supported navigation library by the React Native team. React Router Native is a very thin layer on top of NavigationExperimental that offers React Router’s mental model in a native app. Under the hood, it uses React Router for routing and NavigationExperimental for rendering user components. This is a very powerful combination that makes URLs possible on mobile.

Most apps do not have deep-linking capabilities because implementing it for each screen in your app is a challenging task. Even within apps, users are often forced to take screenshots to share information. And for many, it’s vital that their apps support deep-linking. For example, Yelp goes as far to show a share prompt when users take screenshots of business listings. React Router Native enables developers to implement deep-linking in their apps without putting forth much effort. This can pave the way for a more connected app ecosystem.

That being said, we’re still in the early days of React Native figuring out the right abstractions. Navigation on mobile is a challenging task, and having different flavors is only healthier as the community weighs the pros and cons of each approach rather than second guessing best-practices. So I’m hoping to get the community involved to shape the direction of the project.

Q: Is React Router Native designed to be used with any of the official Navigation components written by the React Native team?

Absolutely. One of the primary goals of the project is that we follow React’s “learn once, write anywhere” principle. So you can use the community maintained components, interpolators and pan responders from React Native, and everything is highly customizable if you need instruct NavigationExperimental to do fancy transition animations, etc.

Q: The React Router team has somewhat famously rewritten their API several times in the last two years, each time introducing several breaking changes. Do you hope to keep your library at parity with React Router, breaking changes and all? Case in point, the V4 release of React Router will introduce an all-new API.

React Router v4 is a complete rewrite. There was a lot of head-scratching on Twitter over the entire new set of breaking changes. Many people thought v4 should at best have been released under a different name. I’m not sure if I agree with that sentiment though, I understand where it is coming from. React Router v4 is a preview release, and in my opinion, it’s really hard to argue against replacing a foreign API with simple React components. I do hope to keep the library at parity with React Router, and to be honest, v4’s new everything-is-a-component approach makes the integration even easier. So over the next few weeks I’ll be working on v4 support.

Q: If you were new to React Native, which routing solution would you use? Why?

This is a hard one to answer. Eric Vicenti has done a great job on NavigationExperimental and most of the issues have been sorted out by the community over the last few months. So if you’re familiar with Redux concepts and comfortable writing your own reducers to manage navigation state, NavigationExperimental is a great choice.

One that I’m surprised you didn’t mention that deserves more attention is ExNavigation—another fairly new addition to the brewery. It also uses NavigationExperimental and is maintained by Adam Miskiewicz, Brent Vatne and other awesome members of the Exponent community. It feels a bit tied to the Exponent platform, but runs perfectly fine on React Native and is open source. So you’ve got that.

Finally, If you’re just getting started with React Native and all you need is to be able to click a button and have it transition to a different scene but you don’t want it to get in your way when you need to reach in and apply complex navigational patterns, I strongly recommend you take React Router Native for a spin.




science and technology

Native's Exponent with Charlie Cheever

React Native continues on a development spree in late 2016. With an ambitious two-week release cycle, the framework makes rapid progress towards feature and performance parity with its native Android and iOS equivalents. At the same time, these quick release periods frequently introduce breaking changes, difficulty with setup, and challenges with basic configuration.

Enter Exponent, a tool that promises easier setup, development, and deployment of React Native applications. Rather than being a replacement for React Native, as it is sometimes confused, Exponent augments React Native by dramatically simplifying the development and deployment processes. Whereas basic setup with an Android environment to develop with React Native can take over an hour by hand, even for experienced engineers, Exponent shortens the time to start to “Hello World” to a handful of minutes.

React Native continues on a development spree in late 2016. With an ambitious two-week release cycle, the framework makes rapid progress towards feature and performance parity with its native Android and iOS equivalents. At the same time, these quick release periods frequently introduce breaking changes, difficulty with setup, and challenges with basic configuration.

Enter Exponent, a tool that promises easier setup, development, and deployment of React Native applications. Rather than being a replacement for React Native, as it is sometimes confused, Exponent augments React Native by dramatically simplifying the development and deployment processes. Whereas basic setup with an Android environment to develop with React Native can take over an hour by hand, even for experienced engineers, Exponent shortens the time to start to “Hello World” to a handful of minutes.

Exponent’s prime feature is revealed as it’s namesake IDE. The Exponent IDE is development platform for not only developing apps to test in their respective environment simulators, but also simplifies testing them on real devices.

One of the cofounders of Exponent, Charlie Cheever, agreed to answer a few questions about Exponent and its purpose in the community.


Hi, Charlie. Congrats on the release of Exponent! One of the toughest aspects of Exponent is understanding what its purpose is. What is the primary goal of Exponent?

Thanks :)

Before I worked on mobile software, I spent about 15 years making websites. When I started working on the Quora iPhone app and Android app, it felt like time traveling back to 1993. So many things to worry about that have nothing to do with the product you want to build.

One thing we’re trying to do with Exponent is making it as easy to develop native mobile apps as it is to make websites, or even easier! I think about how I learned to build software as a kid–making games on my TI-85 and making Hypercard stacks–and I want to make it so that the middle school kids of today can make cool stuff for themselves and their friends.

Basic environment setup of the iOS and Android simulators for developing React Native apps is commonly cited as a headache by new developers. What does Exponent do to alleviate this pain?

The biggest thing that Exponent does is take care of everything related to native code for you. So you don’t need to know Swift/Obj-C/Java or even have Xcode or Android Studio to be able to write React Native apps. You write just JavaScript and Exponent has everything else already setup for you.

Since you don’t write any native code with Exponent, just JavaScript, Exponent has a lot of the most popular native modules built in. Native maps, push notifications, Facebook and Google login, camera and camera roll access, contacts, TouchID, and a native video player are all included among other things. We’re always adding more of these as well. We just added full OpenGL support last week and did a game jam and made some mini games with it and are adding sound soon.

We sometimes talk about Exponent as being like Rails for React Native. You could write a website in Ruby on your own. but Rails sets up a bunch of sensible things right off that bat that work together in a coherent way and we kind of do the same thing for React Native. Exponent includes instant app updating as a default, so you can deploy new code and assets with one command in seconds, even faster than most websites can be deployed.

Even after getting set up with the Android and iOS simulators, testing a React Native app on a real phone can still be a challenge. How does Exponent make it easier to share apps in progress with would-be users?

You can actually open any Exponent project that you’re working on in our development app right away. When you develop with Exponent, you get a URL for your project, and you can open that URL on any phone with the Exponent developer app which you can download from the iOS App Store or Google Play Store. You don’t need to jack your phone into your computer–just open the URL.

Another really cool thing about this is that, if you’re working with someone else, you can just send them the URL and they can open it on their phone as well, even if they are halfway around the world.

We’ve done a bunch of work to make this pretty nice, like having console.log work even if the phone running your code isn’t plugged into your computer. And you can, of course, open your project on the iOS Simulator or an Android Emulator as well if you prefer.

I know you mentioned a lot of people have trouble getting React Native setup on Android especially. With Exponent, every project works on both iOS and Android from the start and you never have to deal with Android Studio, so the process of getting going is much easier.

What type, or genre, of application would be a good fit with React Native and Exponent?

I would actually use React Native for almost any mobile app at this point. Doing development the traditional way (writing Swift/Java/Obj-C code) is just too hard to iterate on when you consider the slowness of the code-compile-copy-run loop and the fact that you have to write your app twice (and then keep it in sync!). The other thing that is an absolutely huge deal here but is sometimes overlooked is the layout engine. It’s much easier to build and change a layout in React Native’s Flexbox than any of the UI libraries that I’ve seen for Java/Swift/Obj-C.

And if you need to do something really intense, like Snapchat live video filters, you can just write your own code as a native module and write the rest of your app in JS.

I would use Exponent for anything I could because it just saves a lot of time and headaches since you don’t need to deal with Android Studio or Xcode. Some people don’t know that you can turn an Exponent project into an app store app for iOS or for Android with just one command.

In general, Exponent will work for you in pretty much every case where just having a mobile website is one of the things that you’re considering. The features are pretty equivalent except that Exponent apps feel like native apps and mobile apps still feel like mobile web apps.

The main reason not to use Exponent is if you have some custom native code that you need that isn’t included with Exponent. The most common reasons that people can’t use Exponent are if they need use Bluetooth or HealthKit or something else low level that isn’t built in to Exponent; or if they need to integrate into an existing project (though we are working right now on a solution that will let you do this).

The exception to all this is games. If you are making a mobile game, Unity is probably the best choice for you. But we did add OpenGL support to Exponent recently and had a game jam and I was surprised at how good some of the entries were, so I think that might change.

TL;DR: For apps that aren’t games, always use React Native (if you need to do something super custom, just do it as a native module). If you can, use Exponent (you can most of the time but check our docs to make sure we’re not missing anything you need).

One aspect of React Native that seems to be undergoing constant flux is its solution for navigation. Between the built in Navigators and open source solutions, do you have any thoughts on an ideal solution for navigation?

Short version: I think you should use Ex-Navigation that Adam Miskiewicz (skevy) and Brent Vatne on our team wrote. Skevy in particular has been thinking about navigation in mobile apps and React Native for a long time. Using Ex-Navigation is definitely a better idea than Navigator or NavigatorIOS.

To make things confusing, there is also NavigatorExperimental (yes, that’s different from Ex-Navigation) and ExNavigator (which was made by James Ide and Ex-Navigation is based on). The good news is that everyone working on these problems got together and decided to merge them all together. I don’t know how long that is going to take but it will probably be released sometime in the next few months under the name React Navigation, and that should unify everyone’s efforts!

There is also this other school of thought where some people like to use the platform-specific native code for navigation which is the approach that the Wix Navigator uses. I have a strong personal view that its preferable to write UI components like this in JS because I actually think you want your app to be the same across iOS and Android (they are both just black rectangles with touch screens!) and JS tends to make your code more composable and customizable.

Use Ex-Navigation and keep an eye out for React Navigation! Use JS instead of native for this UI code!

Given the increasingly fast development and deployment times, handling API setup for dealing with data is becoming a large obstacle to React Native apps. Do you have any thoughts about the use of Backend-As-A-Service solutions like Firebase compared to rolling your own API with Node/Express, Rails, or similar?

I don’t have a strongly held view on this right now. There are so many solutions that fit the use cases of people with different needs. We’re seeing things getting easier and easier in every direction that you look.

If you want to write your own code and you’re using JS, you can use something like Zeit’s new now stuff to deploy essentially instantly. If you want a more general purpose solution, Heroku is also really easy. And then of course there is AWS and Google Cloud, etc.

It’s trivially easy for React Native apps to communicate with essentially any backend that uses HTTP/JSON since fetch and JSON.parse are built-in.

If you don’t want to write any code, it seems like Firebase has become the most popular solution since Parse announced its shutdown. One nice thing about Firebase is that you can use their hosted database stuff with React Native using just JS, which means it works just fine with Exponent. Someone wrote up a guide to how to do this here: https://gist.github.com/sushiisumii/d2fd4ae45498592810390b3e05313e5c

Longer term, it seems like something like GraphQL/Relay should become really popular, but that stuff is too hard to setup and use still to be mainstream just yet. I’m not sure whether it will be GraphQL/Relay maturing and getting revised that wins or something else that is slightly different and easy to think about as a developer that comes and beats it, but directionally, it’s definitely right. We built something like this at Quora and it saved a ton of development time.

I would just use whatever you are most comfortable with – almost anything will work! React Native is really similar to the the web in terms of its client capabilities and so I would just think about a React Native or Exponent app as being mostly like a website.




science and technology

Leveraging React for Easy Image Management

React is a good tool when it comes to building flexible and reusable UI components. However, it’s “one of those libraries” that cannot handle all the tasks involved in building a full fleshed UI project. Other supporting tools - such as a recently announced React SDK from Cloudinary - are available to provide solutions that the React core cannot.

In such cases where media (images and videos) becomes a heavy task to handle, Cloudinary simplifies the process with the new React SDK. Let’s build and image library with Cloudinary and React using the Cloudinary’s React SDK.

Prerequisites

The only requirements for using Cloudinary in your existing React project are to install the React SDK and the upload widget. If you do not have an existing React project and want to try these examples, take the following steps:

1. Install Dependencies

We need a minimal amount of dependencies so we can focus on building a media library and not structuring a React app:

{
  "name": "img-library",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "watch": "webpack -d --watch",
    "build": "webpack",
    "serve": "serve ./public"
  },
  "author": "",
  "license": "MIT",
  "devDependencies": {
    "babel-core": "^6.18.2",
    "babel-loader": "^6.2.9",
    "babel-preset-es2015": "^6.18.0",
    "babel-preset-react": "^6.16.0",
    "serve": "^1.4.0",
    "webpack": "^1.14.0"
  },
  "dependencies": {
    "axios": "^0.15.3",
    "cloudinary-react": "^1.0.1",
    "react": "^15.4.1",
    "react-dom": "^15.4.1"
  }
}

React (and React DOM) must be used since we are making a React app. The cloudinary-react dependency is Cloudinary’s React SDK, which we will soon see how it works. axios is a tool for making HTTP requests and, in our case, we will use it request images from the Cloudinary server.

# Install dependencies
npm install

2. Setup Webpack

Webpack is our build tool. Only minimal settings are required to have a build running and our React app compiling:

// ./webpack.config.js
var webpack = require('webpack');
var path = require('path');

var BUILD_DIR = path.resolve(__dirname, 'public');
var APP_DIR = path.resolve(__dirname, 'src');

var config = {
    entry: APP_DIR + '/index.jsx',
    output: {
        path: BUILD_DIR,
        filename: 'bundle.js'
    },
    module : {
        loaders : [
            {
                test : /.jsx?/,
                include : APP_DIR,
                loader : 'babel'
            }
        ]
    }
};

module.exports = config;

Basic configuration - an entry, output and loaders to handle the React .jsx files.

3. Entry Points

We need to create an entry point, as we specified in the Webpack configuration, and another entry point for the browser, which is an index.html file:

// ./src/index.jsx
import React, { Component } from 'react';
import { render } from 'react-dom';

class Main extends Component {
    render() {
        return (
           <div className="main">
               <h1>Scotchage</h1>
           </div>
        );
    }
}

render(<Main />, document.getElementById('container'));
<!-- ./public/index.html -->
<html>
<head>
    <!--Stylesheet-->
    <link rel="stylesheet" href="style.css">
    <meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
    <!--Container for React rendering-->
    <div id="container"></div>
    <!--Bundled file-->
    <script src="bundle.js"></script>
</body>
</html>

4. Create Cloudinary Account

You need a Cloudinary account to continue with these examples. Sign up for free and store your credentials safely as shown on the dashboard:

Uploading Images

Before using the React SDK to deliver images from the Cloudinary servers, let’s use the awesome Cloudinary upload widget to upload images. First, we need to add this widget to our index.html:

<!-- ./public/index.html -->
<html>
<head>
   . . .
</head>
<body>
    . . .
    <!-- UPLOAD WIDGET -->
    <script src="//widget.cloudinary.com/global/all.js" type="text/javascript"></script>
    <script src="bundle.js"></script>
</body>
</html>

Next, we create a button, attach an event to it and upload an image once the button is clicked:

import React, { Component } from 'react';
import { render } from 'react-dom';

class Main extends Component {

    uploadWidget() {
        cloudinary.openUploadWidget({ cloud_name: 'CLOUD_NAME', upload_preset: 'PRESET', tags:['xmas']},
            function(error, result) {
                console.log(result);
            });
    }
    render(){
        return (
            <div className="main">
                <h1>Galleria</h1>
                <div className="upload">
                    <button onClick={this.uploadWidget.bind(this)} className="upload-button">
                        Add Image
                    </button>
                </div>
            </div>

        );
    }
}

render(<Main />, document.getElementById('container'));

The uploadWidget member method is the handler invoked by the click event to handle our image upload by calling cloudinary.openUploadWidget. openUploadWidget takes a config object and the upload callback handler. The config object must have at least cloud_name and upload_preset properties with valid values. You can read more about Cloud Names and Upload Presets.

Delivering Images with SDK

The Cloudinary React SDK has three major components, Image, CloudinaryContext and Transformation:

  • Image: This component is responsible for the actual delivery of images. It takes the image ID and asks the server for this image. When the image is provided, it is also responsible for painting the image on the browser.
  • Transformation: This component is used to apply transformations to images delivered with Image.
  • CloudinaryContext: You can specify Cloudinary configuration for each image on the Image component. This can be tedious when you are dealing with multiple images. CloudinaryContext allows you to apply configuration to a group of Images.

Most times you would end up with a structure like this:

<CloudinaryContext>
    <Image>
        <Transformation />
        <Transformation />
    </Image>
    <Image>
        <Transformation />
    </Image>
</CloudinaryContext>

Back to our demo app, we can request an image from the Cloudinary server and display it with the following components:

import React, { Component } from 'react';
import axios from 'axios';
import { CloudinaryContext, Transformation, Image } from 'cloudinary-react';
import { render } from 'react-dom';

class Main extends Component {
    constructor(props) {
        super(props);
        this.state = {
            gallery: []
        }
    }
    componentDidMount() {
    // Request for images tagged xmas       
axios.get('http://res.cloudinary.com/christekh/image/list/xmas.json')
            .then(res => {
                console.log(res.data.resources);
                this.setState({gallery: res.data.resources});
            });
    }
    uploadWidget() {
       // . . .
    }
    render(){
        return (
            <div className="main">
                <h1>Galleria</h1>
                <div className="gallery">
                    <CloudinaryContext cloudName="CLOUDNAME">
                        {
                            this.state.gallery.map(data => {
                                return (
                                    <div className="responsive" key={data.public_id}>
                                        <div className="img">
                                            <a target="_blank" href={`http://res.cloudinary.com/christekh/image/upload/${data.public_id}.jpg`}>
                                                <Image publicId={data.public_id}>
                                                    <Transformation
                                                        crop="scale"
                                                        width="300"
                                                        height="200"
                                                        dpr="auto"
                                                        responsive_placeholder="blank"
                                                    />
                                                </Image>
                                            </a>
                                            <div className="desc">Created at {data.created_at}</div>
                                        </div>
                                    </div>
                                )
                            })
                        }
                    </CloudinaryContext>
                    <div className="clearfix"></div>
                </div>
            </div>

        );
    }
}

render(<Main />, document.getElementById('container'));

Take one more look at the upload code:

 cloudinary.openUploadWidget({ cloud_name: 'christekh', upload_preset: 'idcidr0h', tags:['xmas']},
            function(error, result) {
            . . .

Each image is tagged with xmas, which serves as a way to request images with this tag as a collection. This is exactly what we are using the axios library to do when the component mounts:

axios.get('http://res.cloudinary.com/CLOUDNAME/image/list/xmas.json')
            .then(res => {
                console.log(res.data.resources);
                this.setState({gallery: res.data.resources});
            });

axios uses promises, so whenever the promise resolves in our case, we have a payload of images. We take advantage of React state to update our UI with the fetched resources.

Down to rendering, we configure the CloudinaryContext with our cloud_name, iterate over the gallery state that stores the images and displays them using the Image component. We also apply few transformations using the Transformation component.

For security reasons, Cloudinary will not allow you to make such request from the client unless you tell it to. The best way to go is to use the admin API via a backend SDK and then send the resource list to the client.

Updating State with New Uploads

We are able to upload images and request for images to be displayed on the user’s browsers. Here is how we update the displayed images instantly when the user uploads a new image:

uploadWidget() {
        let _this = this;
        cloudinary.openUploadWidget({ cloud_name: 'CLOUDNAME', upload_preset: 'PRESET', tags:['xmas']},
            function(error, result) {
            // Update gallery state with newly uploaded image
                _this.setState({gallery: _this.state.gallery.concat(result)})
            });
    }

Rather than logging the uploaded image information to the console, we update the gallery state, which bears the list of requested images, by concatenating the uploaded result to the gallery.

Image Management Simplified

Image uploads, transformation and delivery has never been easier. These tasks have been a serious challenge for developers. Cloudinary has created a way to abstract all this hard work, enabling you to simply plug and play.




science and technology

The Diverse React Navigation Ecosystem

The routing ecosystem around React and React Native is quite different. One is characterized by a strong incumbent, and the other is plagued by rapid change.

React JS

No question about it, React Router (ReactTraining/react-router) is king here. They have the benefit of several years of active development, along with an active community submitting PR’s, fixes, etc. Supplement that with myriad tutorials and how-to’s and you end up with a well-supported, stable product. To be fair, React Router has suffered some major API upsets, and is nearly the poster-child for javascript fatigue but the maintainers have declared their lasting support for a few specific versions, which means you can plop down with V3 and be good to go for the next 12 to 24 months.

React Native

Ok, this is where things start to get really, really crazy. The most important thing to keep in mind is that the React Native team has produced three navigation helpers: NavigatorIOS, Navigator, and NavigatorExperimental.

  • NavigatorIOS was quickly deprecated, as it was supported only by (you guessed it) IOS.
  • Navigator is the currently endorsed solution for navigation, at least if you are following the official docs. However, its about to be upset by-
  • NavigationExperimental. This is an updated navigator that has learned some lessons from Navigator, and has some solid integration with Redux. ‘Navigator’ is (or was!) sleighted to be deprecated in favor of NavigationExperimental at some point.

Already you have three ‘official’ navigators supported by the React Native team. The big issue with all three of these is that they are somewhat lightweight and don’t include a lot of common navigation situations out of the box, like sidebars, tab bars, headers, etc. To solve that, the community has introduced…

  • React Native Router Flux (aksonov/react-native-router-flux). My personal favorite, this is router is based upon NavigationExperimental. You can imagine that the authors looked at NavigationExperimental, realized that everyone would be writing the same wrapper code around it, and so created this project.
  • React Native Router (t4t5/react-native-router). Haven’t used it, but it is colossally popular.
  • React Router Native (jmurzy/react-router-native). Strives for API conformity with React Router (the above mentioned one). The great approach here is that they bring in the concept of a URL in native apps, where one doesn’t otherwise exist. This is a great approach that simplifies a lot of common routing situations. Of course, there’s one more big solution that is supposedly going to become the standard:
  • React Navigation (react-community/react-navigation). Seen as a solution that will soon be ‘official’ in the community, it is intended to replaced NavigationExperimental. This package is still in active development, so I expect at least a bit of API upset over the coming months. If you want to use official solutions, go with this, if you want a tried and true solution, go with React Native Router Flux.




science and technology

Component Kits for React Native

You won’t find as many styling solutions for React Native as you will for React JS. This stems from two simple realities:

  1. React Native is a much smaller target for component libraries than traditional CSS frameworks. In other words, Bootstrap CSS can be used with any web framework, whereas component libraries for React Native only work with…you guessed it…React Native.
  2. Customizing React Native styling isn’t the easiest thing in the world. Many apps demand custom styling, which makes component kits not too useful. In addition, it is challenging to customize each and every component, as the flexibility that you gain with traditional CSS on the web doesn’t carry over easily to component libraries.

With that said, here are a few options.

You won’t find as many styling solutions for React Native as you will for React JS. This stems from two simple realities:

  1. React Native is a much smaller target for component libraries than traditional CSS frameworks. In other words, Bootstrap CSS can be used with any web framework, whereas component libraries for React Native only work with…you guessed it…React Native.
  2. Customizing React Native styling isn’t the easiest thing in the world. Many apps demand custom styling, which makes component kits not too useful. In addition, it is challenging to customize each and every component, as the flexibility that you gain with traditional CSS on the web doesn’t carry over easily to component libraries.

With that said, here are a few options.

NativeBase - Essential cross-platform UI components for React Native

A huge collection of components, most of which look quite nice. That’s the plus side. The down side is that some of the components are somewhat buggy. No offense to the library authors, its just the state of the library - it needs a bit of work. For example, here’s an issue I opened a few days ago when I discovered the swipe deck component crashed when only a single data element was provided: DeskSwiper throws on single element lists · Issue #562 · GeekyAnts/NativeBase. The authors fixed it up awfully fast, but, hey, that’s a bug that seems like it could have been caught earlier.


React Native Elements - react-native-community/react-native-elements

This is my personal favorite. The styling is generally platform agnostic; it won’t look out of place using it on either Android or iOS. Each component has simple customization, the docs are solid, and it comes with a good set of icons. This is a no-brainer.


React Native Material Design - react-native-material-design/react-native-material-design

Another solid choice, but mostly only useful for Android. Again, its a bit unsettling to see material design - traditionally a stable of Android devices - on iOS. Besides that, the docs are still a work in progress, as evidenced by the lack of docs for nearly half of the components. Nonetheless, if you’re looking for a material design solution, this is better than nothing. It is also worth noting that the project looks generally unmaintained.


React Native Material Kit - xinthink/react-native-material-kit

Another material design solution, but much better maintained than React Native Material Design. This one has the added benefit of a nicer customization API for creating your own custom components - see the docs on this. It also has some more dynamic components like progress bars and sliders, which you may not see on other frameworks. Anything that helps save you time to build your app is always a solid benefit.


Do Your Own Styling!

If none of these choices float your boat, you can always learn how to style components from scratch yourself. I have a course on Udemy that will teach you how to make perfectly reusable components for your own projects. Check it out here: The Complete React Native and Redux Course - Udemy




science and technology

Using Proxies with Redux Types

One of the most common problems that I run into when using Redux is trying to figure out why an action is not being captured by a reducer. For someone just getting starting with Redux, debugging this issue can be especially overwhelming because of how Redux manages data flow. So before you start pouring over configuration code, or the logic contained in your action creators and reducers, please, make sure your action types are defined and spelled correctly.

One of the most common problems that I run into when using Redux is trying to figure out why an action is not being captured by a reducer. For someone just getting starting with Redux, debugging this issue can be especially overwhelming because of how Redux manages data flow. So before you start pouring over configuration code, or the logic contained in your action creators and reducers, please, make sure your action types are defined and spelled correctly.

In any application that I have built, most bugs that I have run into are simply due to typos. However, the solution to this particular problem is harder to spot because no errors are raised when the application is run. Take a look at the snippet below.

// actionTypes.js

export const FETCH_FILE_REQUEST = 'fetch_file_request';
export const FETCH_FILE_SUCCESS = 'fetch_file_success';
export const FETCH_FILE_FAIL = 'fetch_file_fail';


// filesReducer.js

import {
  FETCH_FILE_REQUEST,
  FETCH_FILE_SUCESS,
  FETCH_FILE_FAIL
} from '../actions/actionTypes';

const filesReducer = (state = {}, action) => {
  switch (action.type) {
    case FETCH_FILE_SUCESS:
      return { ...state, file: action.payload };
    default:
      return state;
  }
}

export default filesReducer;

Assuming we dispatched an action with type FETCH_FILE_SUCCESS, the filesReducer should catch the action before the default case is returned. But what if that is not happening? Where do we start the debugging process. There does not appear to be anything wrong with the code in the reducer; the action type was imported and matches the case in the switch statement. There are no errors in the browser. Where is the issue?

You may have noticed that I misspelled SUCCESS in filesReducer.js, but the reason this can be hard to catch is because importing undefined types does not cause an error, so when we import FETCH_FILE_SUCESS, its value is actually undefined, so our reducer always hits the default case.

It would be nice if the existing import/export system could help us catch this. Unfortunately, since action types are just strings, validating their existence is challenging. Luckily, we have another option.

Enter Proxies

Proxies are a feature of ES2015 that allow us to customize operations on a object. They can be used in many different ways, and you can find some useful examples here and here. For our problem, this example from Mozilla looks promising:

let validator = {
  set: function(obj, prop, value) {
    if (prop === 'age') {
      if (!Number.isInteger(value)) {
        throw new TypeError('The age is not an integer');
      }
      if (value > 200) {
        throw new RangeError('The age seems invalid');
      }
    }

    // The default behavior to store the value
    obj[prop] = value;

    // Indicate success
    return true;
  }
};

let person = new Proxy({}, validator);

person.age = 100;
console.log(person.age); // 100
person.age = 'young'; // Throws an exception
person.age = 300; // Throws an exception

So if proxies can be used to validate that properties assigned to an object are of a certain type and value, we should definitely be able to ensure that our action types are never undefined, or else throw an error that will be easy for us to fix. Let’s refactor our actionTypes.js file.

// actionTypes.js

const types = {
  FETCH_FILE_REQUEST: 'fetch_file_request',
  FETCH_FILE_SUCCESS: 'fetch_file_success',
  FETCH_FILE_FAIL: 'fetch_file_fail'
}

const typeValidator = {
  get(obj, prop) {
    if (obj[prop]) {
      return prop;
    } else {
      throw new TypeError(`${prop} is not a valid action type`);
    }
  }
}

module.exports = new Proxy(types, typeValidator);

First, we define a object containing all our action types. Then we define our validator handler typeValidator. The get method inside our handler is called a trap, and provides access to the properties of a object. If the property we are looking for, an action type, in this case, exists in the types object, return that prop, unmodified. Otherwise, throw an error because the prop does not exist.

Finally, export a new proxy, passing the types object as the target and the typeValidator as the handler. However, it is important to note that the ES2015 module system does not work well with proxies, so module.exports and require() must be used for exporting and importing the types.

Barely any code needs to change in the reducer and action creator files, but in order for the action types to be imported successfully, we just need couple lines of code in a new file:

// actionTypesProxy.js

export const {
  FETCH_FILE_REQUEST,
  FETCH_FILE_SUCCESS,
  FETCH_FILE_FAIL,
} = require('./actionTypes');

// in the reducer and action creator files
// change '../actions/actionTypes' to
// '../actions/actionTypesProxy'

By creating a proxy to verify the existence of an action type, we no longer have to worry about correctly naming a property upon import because an error will be thrown in the browser console as soon as the application starts. So, reduce the number headaches you get when developing an application using Redux and start using proxies.

Interested in learning how to build applications using Redux with ReactJS. Check out this online course! Modern React with Redux




science and technology

Unproven herbal remedy against COVID-19 could fuel drug-resistant malaria, scientists warn

Several African leaders have expressed an interest in a plant-based tonic developed in Madagascar




science and technology

Watch this mosquito-inspired drone light up and avoid a crash

Technology avoids obstacles by sensing air flow disruptions




science and technology

Some comb jellies survive the winter by eating their young

Late-summer cannibalism may help adults make it through tough times




science and technology

India’s push to relax environmental assessment rules amid pandemic draws criticism

Government continues to greenlight projects despite reduced oversight