an

'I feel fresher and healthier' - Hamilton enjoying parts of F1 break

Mercedes driver Lewis Hamilton says he feels certain benefits from his enforced time away from Formula 1.




an

Extra substitutes among temporary law changes when leagues resume

Teams will be allowed to use five substitutes when the season resumes after a Fifa proposal to help with fixture congestion was approved.




an

How Florian Schneider and Kraftwerk influenced five decades of music

The band's influence can be heard in everything from art-rock and hip-hop to trance and house.




an

Can robotaxis ease public transport fears in China?

More self-driving cabs are being launched in China at a time when people are worried about public transport.




an

India coronavirus: Why celebrating Covid-19 'success models' is dangerous

Experts tell the BBC that euphoria over success models runs the risk of people becoming complacent.




an

Dateci Voce: Italian women demand voice in Covid-19 fight

Women post selfies demanding more representation in official bodies dealing with Covid-19.




an

Coronavirus: Belarus WW2 parade defies pandemic and upstages Putin

Belarus goes ahead with a Victory Day parade but in Russia coronavirus forced its cancellation.




an

Coronavirus: How they tried to curb Spanish flu pandemic in 1918

Face masks, fresh air and porridge - how people tried to curb a deadly flu pandemic in 1918.




an

The flamboyant life of 'King and Queen of rock 'n' roll'

The self-styled "king and queen of rock 'n' roll" - who inspired Elvis and The Beatles - dies at 87.




an

Coronavirus: Tesla ordered to keep main US plant closed

It reportedly planned to re-open on Friday, but authorities say this could lead to more virus cases.




an

UK 'to bring in 14-day quarantine' for air passengers

An airline industry body says it has been told coronavirus quarantining will start from the end of May.




an

Slovenia cyclists hold anti-government protest

Thousands accused PM Janez Jansa of using the coronavirus crisis as a pretext to restrict freedoms.




an

Coronavirus: Andy Serkis reads entire Hobbit live online for charity

More than 650,000 watch the Gollum actor narrate Tolkien's fantasy classic in one 11-hour sitting.




an

Magician Roy Horn dies after catching coronavirus

He and his stage partner Siegfried Fischbacher were one of the longest-running acts in Las Vegas.




an

Coronavirus: China offers to help North Korea fight pandemic

President Xi Jinping expresses concern about the threat to its neighbour, and offers to help.




an

Love Bug's creator tracked down to repair shop in Manila

Two decades after the world's first major computer virus, an author finds the perpetrator in Manila.




an

Coronavirus: Can live-streaming save China's economy?

In China, the live-streaming industry has become an important platform for economic recovery.




an

Coronavirus: How does contact tracing work and is my data safe?

Millions in the UK will soon be asked to download an app that helps to limit coronavirus spreading.




an

Coronavirus: Ghana's dancing pallbearers become Covid-19 meme

Social media users have adopted the troupe as a dark-humoured symbol of death in the time of Covid-19.




an

Robot offers help to human co-workers and other tech stories

BBC Click's Jen Copestake looks at some of the best of the week's technology stories.




an

Facebook update crashes TikTok and other rivals

The social network apologised after a software update affects several popular apps on iPhones.




an

Coronavirus: Scam sites selling masks and fake cures taken down

More than 160,000 suspicious emails have been reported to a new scam-busting service in two weeks.




an

Coronavirus: Google ends plans for smart city in Toronto

Sister firm Sidewalk Labs cites Covid-19 as the reason for stepping back from its ambitious plan.




an

Facebook and Google extend working from home to end of year

The tech giants plan to re-open offices soon but will allow staff to work remotely throughout 2020.




an

Coronavirus: 'Plandemic' virus conspiracy video spreads across social media

A slickly-produced "documentary" has exploded across social media, peddling medical misinformation.




an

TileDB 2.0, Scylla 4.0, and CockroachDB raises extra funds

#303 — May 8, 2020

Read on the Web

Database Weekly

Introducing Scylla Open Source 4.0 — Scylla (a Cassandra-compatible NoSQL data store aiming to be the “world’s fastest column-store database”) now provides production-ready lightweight Transactions (LWT), a DynamoDB-compatible API (Alternator), and operator for Kubernetes, and more.

Dor Laor

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

????Live Coding: Guide to Grafana 101 - Getting Started with AlertsJoin us on May 20th to see how to use Grafana’s alerting functionality to get notified about anomalies in your data, dig into root causes, and respond to critical issues. Step-by-step demos + tips = cheaper, more flexible monitoring ✅.

Timescale sponsor

TileDB 2.0 and the Future of Data Science — TileDB is an embeddable storage engine focused on working with dense and sparse multi-dimensional arrays. It’s a C++ library with official Python, R, Java and Go integrations, but it can integrate with other database systems too. 2.0 introduces dataframe support, a new API for R, and support for Google Cloud Storage and Azure Blob Storage.

Stavros Papadopoulos

Time-Series Compression Algorithms, Explained — Delta-delta encoding, Simple-8b, XOR-based compression, and more - these algorithms aren’t magic, but combined they can save over 90% of storage costs and speed up queries. Here’s how they work.

Joshua Lockerman and Ajay Kulkarni

CockroachDB Creators Raise $87 Million of New Investment — Quite a raise and quite a valuation in these times for the creators of CockroachDB, a popular distributed SQL database.

Cockroach Labs

The Big Cloud Data Boom Gets Even Bigger, Thanks to COVID-19? — It’s not like the cloud was doing badly beforehand, but the pandemic is apparently encouraging companies to virtualize as much of their operations as possible.

Datanami

MongoDB Is Easy. Now Make It Powerful. Free Download for 30 Days. — Using MongoDB Atlas? Studio 3T is the professional GUI and IDE that unlocks the power you need.

Studio 3T sponsor

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using AWS API Gateway to Run Database Queries — API Gateway is commonly used to hook up HTTP endpoints to AWS Lambda functions but did you know it can directly connect to DynamoDB? (Or any AWS service that lets you query over the AWS API, so not RDS.)

Renato Byrro

How to Remain Agile with DynamoDB — Amazon DynamoDB delivers performance at scale but at a cost to flexibility (particularly early on in the development cycle when your eventual access patterns aren’t always known) – there are some mitigations, however.

Rob Cronin

Jobs

DevOps Engineer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Tooling

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

AvionDB: A Decentralised Database with MongoDB-like Developer Interface — An admittedloy ‘alpha stage’ database system built on top of OrbitDB, a serverless, peer-to-peer database that uses IFPS for storage and implements the core decentralized database logic/protocol.

Dappkit

mssql-cli, a CLI Tool to Manage SQL Server, Now on macOS and Linux — mssql-cli is a tool for working with SQL Server from the command line, complete with Intellisense, syntax highlighting, and paging.

Alan Yu (Microsoft)




an

Coronavirus: How do you social distance in schools?

If pupils are allowed to return to school after the lockdown, how would they keep 2m apart?




an

Coronavirus schools return: Can you really keep children 2m apart?

What's it like in a school that has re-opened? Denmark and Germany show how it might look.




an

Coronavirus: Thanking healthcare workers worldwide

Some of the ways that people have expressed thanks to workers on the frontline against Covid-19.




an

Coronavirus: I got a life-changing opportunity in lockdown

Ana Carmona chronicled her month in quarantine with her family in NYC, including when she got some big news.




an

Coronavirus and climate change a ‘double crisis’

Many activists have had to stop their usual work due to the pandemic. Here's how they're responding.




an

Worst song possible plays as Trump tours mask plant

As the president touts plans to reopen the economy, Live And Let Die blares over a loudspeaker.




an

How the Covid-19 pandemic is threatening Africa’s wildlife

Park rangers in Africa say the closure of safari tourism is leading to an increase in poaching.




an

Mother of killed Georgia man seeks justice

Wanda Cooper says her son, Ahmaud Arbery, was "hunted down like an animal and killed".




an

Coronavirus: Russian hospital staff 'working without masks'

As coronavirus spreads in the provinces, more and more health workers are getting sick - and dying.




an

ICYMI: Penguin chicks and new dining ideas

Some of the stories from around the world that you may have missed this week.




an

Norfolk Island morepork owls: Major breakthrough for rare species

Two fledglings may have safeguarded the future of the Norfolk Island morepork owl.




an

Let’s hang (Spotify)

Love music? Follow your own tastes? Let’s share. Connect on Spotify.Connect on Last.fm. As a bonus, if we connect on Spotify, you not only get access to An Event Apart’s playlists from the past decade, you also get a preview of the 2020 playlist in progress.

The post Let’s hang (Spotify) appeared first on Zeldman on Web & Interaction Design.




an

Another Blue Beanie Day

Yesterday was the nth annual Blue Beanie Day. (I’ve lost track of what year the standardista holiday started.) I was awake at 1:00 AM on Friday night/Saturday morning, so I tweeted “Happy #BlueBeanieDay,” then slept. No blog post, no prelude—just a past-midnight tweet, over and out. Saturday, once or twice, I checked Twitter and retweeted […]

The post Another Blue Beanie Day appeared first on Zeldman on Web & Interaction Design.




an

A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX.

It’s one thing to seek diverse talent to add to your team, another to retain the people you’ve hired. Why do so many folks we bring in to add depth and breadth of experience to our design and business decision-making process end up leaving? Hear thoughtful, useful answers to this question and other mysteries of […]

The post A panel on accessibility, design inclusion and ethics, hiring and retaining diverse talent, and landing a job in UX. appeared first on Zeldman on Web & Interaction Design.




an

React v16.9.0 and the Roadmap Update

Today we are releasing React 16.9. It contains several new features, bugfixes, and new deprecation warnings to help prepare for a future major release.

New Deprecations

Renaming Unsafe Lifecycle Methods

Over a year ago, we announced that unsafe lifecycle methods are getting renamed:

  • componentWillMountUNSAFE_componentWillMount
  • componentWillReceivePropsUNSAFE_componentWillReceiveProps
  • componentWillUpdateUNSAFE_componentWillUpdate

React 16.9 does not contain breaking changes, and the old names continue to work in this release. But you will now see a warning when using any of the old names:

As the warning suggests, there are usually better approaches for each of the unsafe methods. However, maybe you don’t have the time to migrate or test these components. In that case, we recommend running a “codemod” script that renames them automatically:

cd your_project
npx react-codemod rename-unsafe-lifecycles

(Note that it says npx, not npm. npx is a utility that comes with Node 6+ by default.)

Running this codemod will replace the old names like componentWillMount with the new names like UNSAFE_componentWillMount:

The new names like UNSAFE_componentWillMount will keep working in both React 16.9 and in React 17.x. However, the new UNSAFE_ prefix will help components with problematic patterns stand out during the code review and debugging sessions. (If you’d like, you can further discourage their use inside your app with the opt-in Strict Mode.)

Note

Learn more about our versioning policy and commitment to stability.

Deprecating javascript: URLs

URLs starting with javascript: are a dangerous attack surface because it’s easy to accidentally include unsanitized output in a tag like <a href> and create a security hole:

const userProfile = {
  website: "javascript: alert('you got hacked')",
};
// This will now warn:
<a href={userProfile.website}>Profile</a>

In React 16.9, this pattern continues to work, but it will log a warning. If you use javascript: URLs for logic, try to use React event handlers instead. (As a last resort, you can circumvent the protection with dangerouslySetInnerHTML, but it is highly discouraged and often leads to security holes.)

In a future major release, React will throw an error if it encounters a javascript: URL.

Deprecating “Factory” Components

Before compiling JavaScript classes with Babel became popular, React had support for a “factory” component that returns an object with a render method:

function FactoryComponent() {
  return { render() { return <div />; } }
}

This pattern is confusing because it looks too much like a function component — but it isn’t one. (A function component would just return the <div /> in the above example.)

This pattern was almost never used in the wild, and supporting it causes React to be slightly larger and slower than necessary. So we are deprecating this pattern in 16.9 and logging a warning if it’s encountered. If you rely on it, adding FactoryComponent.prototype = React.Component.prototype can serve as a workaround. Alternatively, you can convert it to either a class or a function component.

We don’t expect most codebases to be affected by this.

New Features

Async act() for Testing

React 16.8 introduced a new testing utility called act() to help you write tests that better match the browser behavior. For example, multiple state updates inside a single act() get batched. This matches how React already works when handling real browser events, and helps prepare your components for the future in which React will batch updates more often.

However, in 16.8 act() only supported synchronous functions. Sometimes, you might have seen a warning like this in a test but could not easily fix it:

An update to SomeComponent inside a test was not wrapped in act(...).

In React 16.9, act() also accepts asynchronous functions, and you can await its call:

await act(async () => {
  // ...
});

This solves the remaining cases where you couldn’t use act() before, such as when the state update was inside an asynchronous function. As a result, you should be able to fix all the remaining act() warnings in your tests now.

We’ve heard there wasn’t enough information about how to write tests with act(). The new Testing Recipes guide describes common scenarios, and how act() can help you write good tests. These examples use vanilla DOM APIs, but you can also use React Testing Library to reduce the boilerplate code. Many of its methods already use act() internally.

Please let us know on the issue tracker if you bump into any other scenarios where act() doesn’t work well for you, and we’ll try to help.

Performance Measurements with <React.Profiler>

In React 16.5, we introduced a new React Profiler for DevTools that helps find performance bottlenecks in your application. In React 16.9, we are also adding a programmatic way to gather measurements called <React.Profiler>. We expect that most smaller apps won’t use it, but it can be handy to track performance regressions over time in larger apps.

The <Profiler> measures how often a React application renders and what the “cost” of rendering is. Its purpose is to help identify parts of an application that are slow and may benefit from optimizations such as memoization.

A <Profiler> can be added anywhere in a React tree to measure the cost of rendering that part of the tree. It requires two props: an id (string) and an onRender callback (function) which React calls any time a component within the tree “commits” an update.

render(
  <Profiler id="application" onRender={onRenderCallback}>    <App>
      <Navigation {...props} />
      <Main {...props} />
    </App>
  </Profiler>);

To learn more about the Profiler and the parameters passed to the onRender callback, check out the Profiler docs.

Note:

Profiling adds some additional overhead, so it is disabled in the production build.

To opt into production profiling, React provides a special production build with profiling enabled. Read more about how to use this build at fb.me/react-profiling.

Notable Bugfixes

This release contains a few other notable improvements:

  • A crash when calling findDOMNode() inside a <Suspense> tree has been fixed.
  • A memory leak caused by retaining deleted subtrees has been fixed too.
  • An infinite loop caused by setState in useEffect now logs an error. (This is similar to the error you see when you call setState in componentDidUpdate in a class.)

We’re thankful to all the contributors who helped surface and fix these and other issues. You can find the full changelog below.

An Update to the Roadmap

In November 2018, we have posted this roadmap for the 16.x releases:

  • A minor 16.x release with React Hooks (past estimate: Q1 2019)
  • A minor 16.x release with Concurrent Mode (past estimate: Q2 2019)
  • A minor 16.x release with Suspense for Data Fetching (past estimate: mid 2019)

These estimates were too optimistic, and we’ve needed to adjust them.

tldr: We shipped Hooks on time, but we’re regrouping Concurrent Mode and Suspense for Data Fetching into a single release that we intend to release later this year.

In February, we shipped a stable 16.8 release including React Hooks, with React Native support coming a month later. However, we underestimated the follow-up work for this release, including the lint rules, developer tools, examples, and more documentation. This shifted the timeline by a few months.

Now that React Hooks are rolled out, the work on Concurrent Mode and Suspense for Data Fetching is in full swing. The new Facebook website that’s currently in active development is built on top of these features. Testing them with real code helped discover and address many issues before they can affect the open source users. Some of these fixes involved an internal redesign of these features, which has also caused the timeline to slip.

With this new understanding, here’s what we plan to do next.

One Release Instead of Two

Concurrent Mode and Suspense power the new Facebook website that’s in active development, so we are confident that they’re close to a stable state technically. We also now better understand the concrete steps before they are ready for open source adoption.

Originally we thought we would split Concurrent Mode and Suspense for Data Fetching into two releases. We’ve found that this sequencing is confusing to explain because these features are more related than we thought at first. So we plan to release support for both Concurrent Mode and Suspense for Data Fetching in a single combined release instead.

We don’t want to overpromise the release date again. Given that we rely on both of them in production code, we expect to provide a 16.x release with opt-in support for them this year.

An Update on Data Fetching

While React is not opinionated about how you fetch data, the first release of Suspense for Data Fetching will likely focus on integrating with opinionated data fetching libraries. For example, at Facebook we are using upcoming Relay APIs that integrate with Suspense. We will document how other opinionated libraries like Apollo can support a similar integration.

In the first release, we don’t intend to focus on the ad-hoc “fire an HTTP request” solution we used in earlier demos (also known as “React Cache”). However, we expect that both we and the React community will be exploring that space in the months after the initial release.

An Update on Server Rendering

We have started the work on the new Suspense-capable server renderer, but we don’t expect it to be ready for the initial release of Concurrent Mode. This release will, however, provide a temporary solution that lets the existing server renderer emit HTML for Suspense fallbacks immediately, and then render their real content on the client. This is the solution we are currently using at Facebook ourselves until the streaming renderer is ready.

Why Is It Taking So Long?

We’ve shipped the individual pieces leading up to Concurrent Mode as they became stable, including new context API, lazy loading with Suspense, and Hooks. We are also eager to release the other missing parts, but trying them at scale is an important part of the process. The honest answer is that it just took more work than we expected when we started. As always, we appreciate your questions and feedback on Twitter and in our issue tracker.

Installation

React

React v16.9.0 is available on the npm registry.

To install React 16 with Yarn, run:

yarn add react@^16.9.0 react-dom@^16.9.0

To install React 16 with npm, run:

npm install --save react@^16.9.0 react-dom@^16.9.0

We also provide UMD builds of React via a CDN:

<script crossorigin src="https://unpkg.com/react@16/umd/react.production.min.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.production.min.js"></script>

Refer to the documentation for detailed installation instructions.

Changelog

React

  • Add <React.Profiler> API for gathering performance measurements programmatically. (@bvaughn in #15172)
  • Remove unstable_ConcurrentMode in favor of unstable_createRoot. (@acdlite in #15532)

React DOM

React DOM Server

  • Fix incorrect output for camelCase custom CSS property names. (@bedakb in #16167)

React Test Utilities and Test Renderer




an

Building Great User Experiences with Concurrent Mode and Suspense

At React Conf 2019 we announced an experimental release of React that supports Concurrent Mode and Suspense. In this post we’ll introduce best practices for using them that we’ve identified through the process of building the new facebook.com.

This post will be most relevant to people working on data fetching libraries for React.

It shows how to best integrate them with Concurrent Mode and Suspense. The patterns introduced here are based on Relay — our library for building data-driven UIs with GraphQL. However, the ideas in this post apply to other GraphQL clients as well as libraries using REST or other approaches.

This post is aimed at library authors. If you’re primarily an application developer, you might still find some interesting ideas here, but don’t feel like you have to read it in its entirety.

Talk Videos

If you prefer to watch videos, some of the ideas from this blog post have been referenced in several React Conf 2019 presentations:

This post presents a deeper dive on implementing a data fetching library with Suspense.

Putting User Experience First

The React team and community has long placed a deserved emphasis on developer experience: ensuring that React has good error messages, focusing on components as a way to reason locally about app behavior, crafting APIs that are predictable and encourage correct usage by design, etc. But we haven’t provided enough guidance on the best ways to achieve a great user experience in large apps.

For example, the React team has focused on framework performance and providing tools for developers to debug and tune application performance (e.g. React.memo). But we haven’t been as opinionated about the high-level patterns that make the difference between fast, fluid apps and slow, janky ones. We always want to ensure that React remains approachable to new users and supports a variety of use-cases — not every app has to be “blazing” fast. But as a community we can and should aim high. We should make it as easy as possible to build apps that start fast and stay fast, even as they grow in complexity, for users on varying devices and networks around the world.

Concurrent Mode and Suspense are experimental features that can help developers achieve this goal. We first introduced them at JSConf Iceland in 2018, intentionally sharing details very early to give the community time to digest the new concepts and to set the stage for subsequent changes. Since then we’ve completed related work, such as the new Context API and the introduction of Hooks, which are designed in part to help developers naturally write code that is more compatible with Concurrent Mode. But we didn’t want to implement these features and release them without validating that they work. So over the past year, the React, Relay, web infrastructure, and product teams at Facebook have all collaborated closely to build a new version of facebook.com that deeply integrates Concurrent Mode and Suspense to create an experience with a more fluid and app-like feel.

Thanks to this project, we’re more confident than ever that Concurrent Mode and Suspense can make it easier to deliver great, fast user experiences. But doing so requires rethinking how we approach loading code and data for our apps. Effectively all of the data-fetching on the new facebook.com is powered by Relay Hooks — new Hooks-based Relay APIs that integrate with Concurrent Mode and Suspense out of the box.

Relay Hooks — and GraphQL — won’t be for everyone, and that’s ok! Through our work on these APIs we’ve identified a set of more general patterns for using Suspense. Even if Relay isn’t the right fit for you, we think the key patterns we’ve introduced with Relay Hooks can be adapted to other frameworks.

Best Practices for Suspense

It’s tempting to focus only on the total startup time for an app — but it turns out that users’ perception of performance is determined by more than the absolute loading time. For example, when comparing two apps with the same absolute startup time, our research shows that users will generally perceive the one with fewer intermediate loading states and fewer layout changes as having loaded faster. Suspense is a powerful tool for carefully orchestrating an elegant loading sequence with a few, well-defined states that progressively reveal content. But improving perceived performance only goes so far — our apps still shouldn’t take forever to fetch all of their code, data, images, and other assets.

The traditional approach to loading data in React apps involves what we refer to as “fetch-on-render”. First we render a component with a spinner, then fetch data on mount (componentDidMount or useEffect), and finally update to render the resulting data. It’s certainly possible to use this pattern with Suspense: instead of initially rendering a placeholder itself, a component can “suspend” — indicate to React that it isn’t ready yet. This will tell React to find the nearest ancestor <Suspense fallback={<Placeholder/>}>, and render its fallback instead. If you watched earlier Suspense demos this example may feel familiar — it’s how we originally imagined using Suspense for data-fetching.

It turns out that this approach has some limitations. Consider a page that shows a social media post by a user, along with comments on that post. That might be structured as a <Post> component that renders both the post body and a <CommentList> to show the comments. Using the fetch-on-render approach described above to implement this could cause sequential round trips (sometimes referred to as a “waterfall”). First the data for the <Post> component would be fetched and then the data for <CommentList> would be fetched, increasing the time it takes to show the full page.

There’s also another often-overlooked downside to this approach. If <Post> eagerly requires (or imports) the <CommentList> component, our app will have to wait to show the post body while the code for the comments is downloading. We could lazily load <CommentList>, but then that would delay fetching comments data and increase the time to show the full page. How do we resolve this problem without compromising on the user experience?

Render As You Fetch

The fetch-on-render approach is widely used by React apps today and can certainly be used to create great apps. But can we do even better? Let’s step back and consider our goal.

In the above <Post> example, we’d ideally show the more important content — the post body — as early as possible, without negatively impacting the time to show the full page (including comments). Let’s consider the key constraints on any solution and look at how we can achieve them:

  • Showing the more important content (the post body) as early as possible means that we need to load the code and data for the view incrementally. We don’t want to block showing the post body on the code for <CommentList> being downloaded, for example.
  • At the same time we don’t want to increase the time to show the full page including comments. So we need to start loading the code and data for the comments as soon as possible, ideally in parallel with loading the post body.

This might sound difficult to achieve — but these constraints are actually incredibly helpful. They rule out a large number of approaches and spell out a solution for us. This brings us to the key patterns we’ve implemented in Relay Hooks, and that can be adapted to other data-fetching libraries. We’ll look at each one in turn and then see how they add up to achieve our goal of fast, delightful loading experiences:

  1. Parallel data and view trees
  2. Fetch in event handlers
  3. Load data incrementally
  4. Treat code like data

Parallel Data and View Trees

One of the most appealing things about the fetch-on-render pattern is that it colocates what data a component needs with how to render that data. This colocation is great — an example of how it makes sense to group code by concerns and not by technologies. All the issues we saw above were due to when we fetch data in this approach: upon rendering. We need to be able to fetch data before we’ve rendered the component. The only way to achieve that is by extracting the data dependencies into parallel data and view trees.

Here’s how that works in Relay Hooks. Continuing our example of a social media post with body and comments, here’s how we might define it with Relay Hooks:

// Post.js
function Post(props) {
  // Given a reference to some post - `props.post` - *what* data
  // do we need about that post?
  const postData = useFragment(graphql`
    fragment PostData on Post @refetchable(queryName: "PostQuery") {
      author
      title
      # ...  more fields ...
    }
  `, props.post);

  // Now that we have the data, how do we render it?
  return (
    <div>
      <h1>{postData.title}</h1>
      <h2>by {postData.author}</h2>
      {/* more fields  */}
    </div>
  );
}

Although the GraphQL is written within the component, Relay has a build step (Relay Compiler) that extracts these data-dependencies into separate files and aggregates the GraphQL for each view into a single query. So we get the benefit of colocating concerns, while at runtime having parallel data and view trees. Other frameworks could achieve a similar effect by allowing developers to define data-fetching logic in a sibling file (maybe Post.data.js), or perhaps integrate with a bundler to allow defining data dependencies with UI code and automatically extracting it, similar to Relay Compiler.

The key is that regardless of the technology we’re using to load our data — GraphQL, REST, etc — we can separate what data to load from how and when to actually load it. But once we do that, how and when do we fetch our data?

Fetch in Event Handlers

Imagine that we’re about to navigate from a list of a user’s posts to the page for a specific post. We’ll need to download the code for that page — Post.js — and also fetch its data.

Waiting until we render the component has problems as we saw above. The key is to start fetching code and data for a new view in the same event handler that triggers showing that view. We can either fetch the data within our router — if our router supports preloading data for routes — or in the click event on the link that triggered the navigation. It turns out that the React Router folks are already hard at work on building APIs to support preloading data for routes. But other routing frameworks can implement this idea too.

Conceptually, we want every route definition to include two things: what component to render and what data to preload, as a function of the route/url params. Here’s what such a route definition might look like. This example is loosely inspired by React Router’s route definitions and is primarily intended to demonstrate the concept, not a specific API:

// PostRoute.js (GraphQL version)

// Relay generated query for loading Post data
import PostQuery from './__generated__/PostQuery.graphql';

const PostRoute = {
  // a matching expression for which paths to handle
  path: '/post/:id',

  // what component to render for this route
  component: React.lazy(() => import('./Post')),

  // data to load for this route, as function of the route
  // parameters
  prepare: routeParams => {
    // Relay extracts queries from components, allowing us to reference
    // the data dependencies -- data tree -- from outside.
    const postData = preloadQuery(PostQuery, {
      postId: routeParams.id,
    });

    return { postData };
  },
};

export default PostRoute;

Given such a definition, a router can:

  • Match a URL to a route definition.
  • Call the prepare() function to start loading that route’s data. Note that prepare() is synchronous — we don’t wait for the data to be ready, since we want to start rendering more important parts of the view (like the post body) as quickly as possible.
  • Pass the preloaded data to the component. If the component is ready — the React.lazy dynamic import has completed — the component will render and try to access its data. If not, React.lazy will suspend until the code is ready.

This approach can be generalized to other data-fetching solutions. An app that uses REST might define a route like this:

// PostRoute.js (REST version)

// Manually written logic for loading the data for the component
import PostData from './Post.data';

const PostRoute = {
  // a matching expression for which paths to handle
  path: '/post/:id',

  // what component to render for this route
  component: React.lazy(() => import('./Post')),

  // data to load for this route, as function of the route
  // parameters
  prepare: routeParams => {
    const postData = preloadRestEndpoint(
      PostData.endpointUrl, 
      {
        postId: routeParams.id,
      },
    );
    return { postData };
  },
};

export default PostRoute;

This same approach can be employed not just for routing, but in other places where we show content lazily or based on user interaction. For example, a tab component could eagerly load the first tab’s code and data, and then use the same pattern as above to load the code and data for other tabs in the tab-change event handler. A component that displays a modal could preload the code and data for the modal in the click handler that triggers opening the modal, and so on.

Once we’ve implemented the ability to start loading code and data for a view independently, we have the option to go one step further. Consider a <Link to={path} /> component that links to a route. If the user hovers over that link, there’s a reasonable chance they’ll click it. And if they press the mouse down, there’s an even better chance that they’ll complete the click. If we can load code and data for a view after the user clicks, we can also start that work before they click, getting a head start on preparing the view.

Best of all, we can centralize that logic in a few key places — a router or core UI components — and get any performance benefits automatically throughout our app. Of course preloading isn’t always beneficial. It’s something an application would tune based on the user’s device or network speed to avoid eating up user’s data plans. But the pattern here makes it easier to centralize the implementation of preloading and the decision of whether to enable it or not.

Load Data Incrementally

The above patterns — parallel data/view trees and fetching in event handlers — let us start loading all the data for a view earlier. But we still want to be able to show more important parts of the view without waiting for all of our data. At Facebook we’ve implemented support for this in GraphQL and Relay in the form of some new GraphQL directives (annotations that affect how/when data is delivered, but not what data). These new directives, called @defer and @stream, allow us to retrieve data incrementally. For example, consider our <Post> component from above. We want to show the body without waiting for the comments to be ready. We can achieve this with @defer and <Suspense>:

// Post.js
function Post(props) {
  const postData = useFragment(graphql`
    fragment PostData on Post {
      author
      title

      # fetch data for the comments, but don't block on it being ready
      ...CommentList @defer
    }
  `, props.post);

  return (
    <div>
      <h1>{postData.title}</h1>
      <h2>by {postData.author}</h2>
      {/* @defer pairs naturally with <Suspense> to make the UI non-blocking too */}
      <Suspense fallback={<Spinner/>}>
        <CommentList post={postData} />
      </Suspense>
    </div>
  );
}

Here, our GraphQL server will stream back the results, first returning the author and title fields and then returning the comment data when it’s ready. We wrap <CommentList> in a <Suspense> boundary so that we can render the post body before <CommentList> and its data are ready. This same pattern can be applied to other frameworks as well. For example, apps that call a REST API might make parallel requests to fetch the body and comments data for a post to avoid blocking on all the data being ready.

Treat Code Like Data

But there’s one thing that’s still missing. We’ve shown how to preload data for a route — but what about code? The example above cheated a bit and used React.lazy. However, React.lazy is, as the name implies, lazy. It won’t start downloading code until the lazy component is actually rendered — it’s “fetch-on-render” for code!

To solve this, the React team is considering APIs that would allow bundle splitting and eager preloading for code as well. That would allow a user to pass some form of lazy component to a router, and for the router to trigger loading the code alongside its data as early as possible.

Putting It All Together

To recap, achieving a great loading experience means that we need to start loading code and data as early as possible, but without waiting for all of it to be ready. Parallel data and view trees allow us to load the data for a view in parallel with loading the view (code) itself. Fetching in an event handler means we can start loading data as early as possible, and even optimistically preload a view when we have enough confidence that a user will navigate to it. Loading data incrementally allows us to load important data earlier without delaying the fetching of less important data. And treating code as data — and preloading it with similar APIs — allows us to load it earlier too.

Using These Patterns

These patterns aren’t just ideas — we’ve implemented them in Relay Hooks and are using them in production throughout the new facebook.com (which is currently in beta testing). If you’re interested in using or learning more about these patterns, here are some resources:

  • The React Concurrent docs explore how to use Concurrent Mode and Suspense and go into more detail about many of these patterns. It’s a great resource to learn more about the APIs and use-cases they support.
  • The experimental release of Relay Hooks implements the patterns described here.
  • We’ve implemented two similar example apps that demonstrate these concepts:

    • The Relay Hooks example app uses GitHub’s public GraphQL API to implement a simple issue tracker app. It includes nested route support with code and data preloading. The code is fully commented — we encourage cloning the repo, running the app locally, and exploring how it works.
    • We also have a non-GraphQL version of the app that demonstrates how these concepts can be applied to other data-fetching libraries.

While the APIs around Concurrent Mode and Suspense are still experimental, we’re confident that the ideas in this post are proven by practice. However, we understand that Relay and GraphQL aren’t the right fit for everyone. That’s ok! We’re actively exploring how to generalize these patterns to approaches such as REST, and are exploring ideas for a more generic (ie non-GraphQL) API for composing a tree of data dependencies. In the meantime, we’re excited to see what new libraries will emerge that implement the patterns described in this post to make it easier to build great, fast user experiences.




an

An intro to making Postgres high availability on Kubernetes

#351 — April 15, 2020

Read on the Web

Postgres Weekly

A Detailed Look at pg_show_plans — A few issues ago we linked to a basic introduction to pg_show_plans – this goes a little further. pg_show_plans lets you look at the execution plans of slow queries in real time as they’re being executed which can help you when troubleshooting.

Kaarel Moppel

Intersecting GPS Tracks to Identify Infected Individuals — I’m not a huge fan of COVID-19 related content, but this is a pretty interesting technique with numerous use cases. Essentially it uses PostGIS to identify overlapping paths.

Florian Nadler

Online Training: Learn PostgreSQL from Home — The remote PostgreSQL Database Administration training course is available at a discounted rate & will be conducted in two different timezones. The course covers day-to-day DBA operations, monitoring, server configurations, and more.

2ndQuadrant PostgreSQL Training sponsor

PostgreSQL's 'Related Projects' — Thanks to Andreas Scherbaum for pointing out a new page on the Postgres site dedicated to projects related to Postgres like the code that runs the Postgres web site, mailing list, build farm, package management system, etc.

PostgreSQL Global Development Group

Authentication Configuration in Postgres (and CockroachDB) — In Postgres, client authentication can be controlled via a ‘HBA’ (host-based authentication) file. It’s not something we see covered very often, so you might find this interesting, particularly as it compares things against CockroachDB.

Raphael ‘kena’ Poss

▶  Easy And Correct High Availability Postgres with Kubernetes — A 50 minute talk from PostgresOpen 2019 that goes all the way ‘from containers up’ until actually doing stuff with Postgres.

Steven Pousty

How To Set Up an Express API Backend Project With Postgres — A pretty extensive walkthrough of creating an HTTP API using Express with Node.js and Postgres on the backend, then deploying it all on Heroku.

Chidi Orji

A Beginners Guide to Basic Indexing in Postgres

James Bannister

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Documenting the Citus Extension to Postgres: An Interview with Joe Nelson — Joe, a.k.a. begriffs, talks about why he works on documentation, why the multi-tenant and real-time analytics tutorials matter, the INSERT..SELECT with repartitioning feature, and what development platform Citus uses for docs.

Citus Data (Microsoft)

Procedural vs Query Approaches for Finding Packages — Explorations of a query that can be used to display which packages are available for a given FreeBSD port. Get your head around the data model and the ideas here apply to all sorts of situations.

Dan Langille

???? Upcoming Events

All in-person events we had listed are cancelled or postponed due to the COVID outbreak, so we're now linking to webinars, livestreams, and similar online events.

If you have any, just hit reply and if it's Postgres related (and either free or not too expensive) we'll include it in a future issue. Just one this week:

???? – requires e-mail address or registration
???? – costs money to participate

???? Seen on Twitter

Saw this tweet and thought it was a pretty neat reminder of the sorts of things we can do with Postgres. Justin kindly let us include it:

Click through to the original tweet if you want to see the code better. Neat use for a generated column!




an

Workloads, acceleration, and making Postgres better

#353 — April 29, 2020

Read on the Web

Postgres Weekly

7 Things That Could Be Improved in Postgres — As 1990s dance pop group D:Ream sang in 1994, Things Can Only Get Better.. including Postgres ???? Luckily these are all ‘nice to have’s but I dare say we’ll see some of them (such as automatic tuning and auto-vacuuming improvements) appear over time.

Kaarel Moppel

How The Citus Distributed Query Executor Adapts to a Postgres Workload — Citus is the popular extension for horizontally scaling Postgres and its query executor has seen some huge updates lately.

Citus Data (Microsoft)

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this pganalyze eBook, we are looking at the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Swarm64 DA 4.0: A Database Acceleration Extension for Postgres — Swarm64 started life as a FPGA-driven way to accelerate Postgres performance, but can now work without FPGAs too. This is not a free product but if you want to give it a run, there’s a trial or it can be spun up from the AWS Marketplace.

Yana Krasteva

Postgres Performance Goalposts — An interesting heuristic from Bruce here on what to do if you expect your connections, queries, or write queries to be above/below certain levels.

Bruce Momjian

A Tale of Password Authentication Methods in Postgres“Let’s say you want to implement a password authentication method in a client/server protocol..” Here’s the story of how Postgres came up with its approaches.

Peter Eisentraut

How to Set application_name When Using psql — As Craig says: “Setting your application name in Postgres is SO USEFUL. It will help a lot for debugging when you’ve got multiple different apps/services connecting to the same database.”

Denish Patel

How to Upgrade Postgres from v11 to v12 on Ubuntu 20.04 — Now that Ubuntu 20.04 is out, this might be on your mind!

Paolo Melchiorre

Working with Amazon Aurora PostgreSQL: What Happened to the Stats? — Apparently there’s a bug with numerous versions of Aurora PostgreSQL that causes certain stats to be lost on restart.

Michael Vitale

Postgres Vision 2020 - Free Online Conference (June 23-24) — Learn how today’s IT leaders are using Postgres. Join from anywhere in the world and listen from 30+ Postgres experts.

EnterpriseDB sponsor

A Deep Dive into PostGIS Nearest Neighbor Search — Take a deep dive into the Postgres and PostGIS internals to find out how K-nearest neighbor accelerates local search.

Martin Davis

My Favorite Postgres Extensions: Part One — A basic high level look at pg_partman and postgres_fdw.

Nawaz Ahmed

Kanel: Generate TypeScript Types from Postgres

Kristian Dupont

Postgres.app: The Easiest Way to Get Started with Postgres on the Mac — I’ve used this for years, it’s super popular, but if there’s just a handful of developers out there who’d benefit from it and don’t know about it, this reminder will be worth it :-) It continues to get very frequent updates.

Jakob Egger, Chris Pastl, and Mattt Thompson

???? Upcoming Events

All in-person events we had listed are cancelled or postponed due to the COVID outbreak, so we're now linking to webinars, livestreams, and similar online events.

If you have any, just hit reply and if it's Postgres related (and either free or not too expensive) we'll include it in a future issue. Just one this week:

  • ???? Postgres Vision 2020 on June 23-24. A full attempt at an online Postgres conference across multiple days with multiple tracks.

???? – requires e-mail address or registration
???? – costs money to participate




an

Stimulus Reflex, and sending thanks to Matz

#498 — April 23, 2020

Read on the Web

Ruby Weekly

Credit: Divina Epiphania / Shutterstock.com

Mining for Malicious Ruby Gems: 700+ Gems Affected — Breathe easy as this was all resolved a month ago (and was too obscure to pay off for the hackers anyway) but a security research team recently found over 700 malicious Ruby gems that were subtle typos/adjustments of more popular gems (e.g. atlas-client vs atlas_client – could you tell which one is real?)

Tomislav Maljic

You Can Now Sponsor Matz on GitHub — I appreciate these are challenging times, but if you’ve ever wanted to give a big thank you to Matz, the creator of Ruby, here’s one way to do it. We’re sponsoring Matz now as without him, this newsletter wouldn’t exist! ???? Alternatively, if you have little to spare, maybe send him a thanks on Twitter?

GitHub Sponsors

Ruby Performance Tips — Here’s a collection of practical tips for improving Ruby performance for better user experiences, brought to you by Raygun. Read the tips here.

Raygun sponsor

Full Text Search in Milliseconds with Rails and Postgres — If you’ve never played with full text search with Postgres and Rails, this is a fine place to start. It covers LIKE/ILIKE, trigrams, and ‘proper’ full text searching. We also get to see how Leigh took a query from taking 130ms down to 7ms.

Leigh Halliday

▶  Introduction to Stimulus Reflex — Stimiulus Reflex makes SPA-type interactions very simple by using ActionCable to render pages and then diffing them on the client.

GoRails

Rails Performance: When is Caching the Right Choice? — Before you say “always”, understand that caching is not free and, if done incorrectly, can even make things worse.

Jonathan Miles

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Ruby Backend Developer (Austria) — We’re seeking mid-level and senior devs to join us and build top-class backend infrastructure for our adidas apps, used by millions. Our stack includes: jRuby, Sinatra, Sidekiq, MySQL, & MongoDB.

Runtastic

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

How to Customize Webpack in Rails Apps — How to go about configuring webpack when tweaking webpacker.yml just isn’t enough.

Ross Kaffenberger

RSpec Given/When/Then with Symbols — An interesting, alternative way to structure a BDD feature in RSpec. I think I prefer the underscores but YMMV.

Caius Durling

Looking Inside a Ruby Gem — Piotr decomposes a .gem file which turns out to just be a collection of gzipped and tarred files, only some of which are the code.

Piotr Murach

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, you will learn about the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Passing Rails Controller Params to SidekiqActionController::Parameters can give Sidekiq issues.

Prathamesh Sonpatki

Catchup Subscriptions with Rails Event Store

Miroslaw Praglowski

Logic-less Ruby Templates with Mustache

David Santangelo

▶  Discussing Ruby for Good with Sean Marcia — Sean talks about founding Ruby For Good (an event about philanthropic Ruby development) and some of the projects it has been responsible for creating.

Ruby Rogues podcast

???? Code and Tools

Impressionist 2.0: A Plugin to Log Impressions in Rails Apps — Impressionist tracks page views and impressions. v2.0 has just dropped but they’re also are looking for new maintainers, so contact them if you want to get involved.

Charlotte Ruby Group

acli 0.3: A Command Line Client for Action Cable — Interesting on two fronts.. first, because it’s an mruby app, and we don’t see many of those, and second, because it lets you play with Action Cable channels in any easier way.

Vladimir Dementyev

Undercover: A Tool to Stop You Shipping Untested Code — It’s like RuboCop but for code coverage rather than code style.

Jan Grodowski

How to Monitor Your Host Metrics Automatically

AppSignal sponsor

Bridgetown: A Modern Ruby (JAMstack) Web Framework — Bridgetown is a new Ruby-based static-site generator based on a fork of Jekyll. It supports plugins and Webpack, so you can use your front-end framework of choice.

Bridgetown

net-ssh 6.0: A Pure Ruby Implementation of the SSH2 Client Protocol — Yes, you can write programs that invoke and interact with processes on remote servers, via SSH2, all in Ruby.

Buck, Fazekas, et al.




an

An interview with Ruby ETL expert Thibaut Barrère

#499 — April 30, 2020

Read on the Web

???? Occasionally we run interviews in Ruby Weekly and we're back with another one.. with long time Rubyist and Kiba maintainer, Thibaut Barrère. Be sure to check out the bottom of this issue to read it, especially if you ever run ETL jobs with Ruby! ????

Ruby Weekly

▶  Let's Build a Twitter Clone in 10 Minutes with Rails, CableReady, and StimulusReflex — You know that cloning Twitter in 10 minutes is impossible, but what about the core mechanism of the idea? After a slow first minute, this video does a pretty good job of showing off some techniques you might not have used before.

Nate Hopkins

discuss.rubyonrails: The Rails Project Discussion Forum — Basically a Web version of the Rails mailing lists and a worthwhile place to head if you want to suggest features, ask questions, etc.

Ruby on Rails Discussions

Easy Rails Deployments — Deploy your Ruby, Rails, Sinatra, and Rack application to any cloud or server. Cloud 66 offers a scalable Heroku alternative that doesn't lock you in. Try it free and get extra $66 free credits with the code: 'Ruby-Weekly'.

Cloud 66 sponsor

sequel-activerecord-adapter: Allows Sequel to Reuse an ActiveRecord Connection — If you want to use more Sequel or migrate in that direction, this makes it easy.

Janko Marohnić

Ruby Adds Experimental Support for 'End-Less' Method Definitions — We’ve touched on this in a previous issue, but if you fancy a proper blog post with examples, this is more accessible than digging through feature tracker discussions. I’m not a fan of this syntax myself yet, but Prateek does a good job of selling it.

Prateek Choudhary

CableReady: Trigger Client-Side DOM Changes from Server-Side Ruby — If you skipped the video above because it’s a video (and I know many of you do ????) CableReady is still worth checking out. It aims to “complete the ActionCable story” by providing a way to directly interact with clients over ActionCable WebSockets. The docs will help you get the idea.

Hopsoft

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Security Engineer (Remote) — Are you an engineer with experience in Rails and/or Go? Join our team and help secure our apps and cloud infrastructure.

Shogun

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

A Practical Use for PStorePStore is one of the older parts of Ruby’s stdlib and lets you persist (and restore) a Ruby hash to disk. I haven’t seen it in Ruby code for years but Arkency have found a neat, modern use case.

Paweł Pacana

▶  Drag and Drop Sortable Lists with Rails and Stimulus JS — How to wire up drag and drop lists with a Rails app using Sortable, Stimulus and acts_as_list.

Go Rails

Setting Up Multi-Factor Authentication for RubyGems.org — If you’re a registered user of the official Ruby Gems repository, you should have had an email this week about securing your account using 2FA. This is just my own reminder that this is a good idea especially if you publicly publish widely used gems! :-)

Rubygems.org

Let’s Explore Big-O Notation With Ruby ???? — Learn to use Big-O to look at an algorithm and easily discern its efficiency, without having to run a profiling tool.

Honeybadger sponsor

▶  Discussing Docker and Kubernetes with Kelsey Hightower — A worthwhile show to listen to if Kubernetes and Docker intimidate you but you want to know a little more. Kelsey is good at breaking these things down into understandable pieces.

Rails with Jason Podcast podcast

Rails System Tests in Docker — We’re seeing an uptick in articles about system tests in Rails. Here’s how to integrate them into your development Docker setup.

Hint.io

The Difference Between System Specs and Feature Specs — If you’ve felt the difference between RSpec’s “feature specs” and “system specs” is quite subtle, this explanation will help.

Jason Swett

Why Rubyists Should Consider Learning Go — If you want compilation and a type system, Crystal is probably a better fit for Rubyists, but Go is undoubtedly a neat language and ecosystem (and if you do end up in the Go world, check out our Go weekly! ????)

Ayooluwa Isaiah

Building a Ruby CLI with Thor

Daniel Gómez

???? Code and Tools

git curate: Peruse and Delete git Branches Ergonomically — Got a repo cluttered with branches here and there? git curate aims to cure the pains of getting those branches back under control.

Matt Harvey

MessageBus: A Reliable and Robust Messaging Bus for Ruby and Rack

Sam Saffron

Are You Spending Too Much on Heroku?

Rails Autoscale sponsor

bootstrap_form: A Rails Form Builder for Bootstrap v4-Style Forms

Bootstrap Ruby

ActiveModelAttributes: The Active Record Attributes API, but for Active Model — Brings some of the goodies of the Rails 5 Active Record attributes API to ActiveModel too. 1.6.0 just dropped.

Karol Galanciak

???? A Q&A with…
Thibaut Barrère
Creator of Kiba ETL

Thibaut Barrère is a long-time Rubyist and data engineer who built and maintains the popular Extract, Transform, and Load (ETL) framework Kiba. We asked him some questions about his work:

What inspired you to create Kiba?

A lot of my work since ~2005 has been focused on data integration (making systems speak together), data aggregation etc. I sometimes used GUI-based tools like Microsoft SSIS which, while powerful, are quite far from the coding experience.

I was already using Ruby at that time, and was happy to discover activewarehouse-etl (maintained by Anthony Eden, who runs DNSimple now) providing a Ruby DSL to declare data pipelines. I used it for a while with very good success to implement data extractions and business intelligence ETL, and ultimately took over the maintenance.

In the long run, though (as I explained in a recent Paris.rb talk), the balance between the the cost of OSS maintenance and the usability for my billable and non-billable use-cases proved to be not good enough, which made me decide to stop the maintenance, sadly.

I still wanted to use Ruby to write data pipelines, but I needed to cover more use-cases and reduce the OSS maintenance burden at the same time. This ultimately led me to write and share (in 2015) Kiba ETL, a focused DSL for declarative processing, matching those criterias.

Do you find yourself adding more features while in quarantine?

Before the quarantine, I directed my Kiba bandwith and focus on finalizing Kiba v3 and rewriting the documentation from scratch, to properly encourage best practices I’ve been discovering. I also created experimental branches for Ruby 2.7/2.8 keywords.

During the quarantine, I've reduced client work and OSS work too, to focus on shipping Kiba Pro v2 (which I’ll announce shortly officially). I’ve extracted and generalized (from real-life projects) very useful components, such as a “batch SQL lookup” (useful when replacing relationships keys during data migrations and datawarehouses sync code, in batch rather than row by row), a “file lock” to ensure a single job runs at once, and a “parallel transform” to achieve easy concurrency for things like HTTP queries.

What's the wildest ETL that you've encountered?

Getting the data out of a system which is actively not acting in that direction is always a bit wild.. One can see all types of fancy stuff on the field. For instance, it is not uncommon to have an ETL process start a headless browser, jump through pages, just to get to the CSV/PDF/Excel file that you will then use as your data source!

You can also end up having to figure out ways to read or write very old file formats at times. Recently I wrote a Kiba component to generate a COBOL delimited file, for instance. In large companies, a very widely used I/O is good old SFTP, far away from modern APIs and formats.

Can you tell us how to say your last name? ????

I had to deploy a page to my blog to answer that question properly ????. You’ll find out how to say my name here.

Merci Thibaut!

You can read some of Thibaut's posts on his blog and find out more about Kiba ETL here.




an

A transpiler for futuristic Ruby, and the RailsConf 2020 videos

#500 — May 7, 2020

Read on the Web

???? Welcome to issue 500! A bit of an arbitrary milestone but thanks to you all :-)

Ruby Weekly

Ruby Next: Make All Rubies Quack Alike — Ruby Next is a Ruby-to-Ruby transpiler that allows you to use the latest features of Ruby in previous versions without monkey patching or refinements. Could this be how experimental features are released going forward?

Vladimir Dementyev

Ruby 3 'Guilds' Proposal Now Called Ractor — This documentation is in Japanese (though the source code examples are easy to follow) but the news is that the new, proposed concurrency mechanism for Ruby 3 called Guilds (explained here) has been renamed to Ractor (as in ‘Ruby actors’, Ruby’s take on the actor model).

Koichi Sasada

Don’t Do Auth From Scratch. Focus On Your App — Spend less time on authentication and authorization and more time developing your awesome app. Auth built for <devs>. Download our community edition for free.

FusionAuth sponsor

Take the 2020 Ruby on Rails Survey — This is the sixth outing for Planet Argon’s survey which began in 2009. We try and support it each time as the results always make for interesting reading (see 2018’s results). Participate and become data ????

Planet Argon Team

???? RailsConf 2020 Videos

If you recall, RailsConf 2020 was cancelled in its in-person form to be replaced by a 'couch edition'. This has been taking place and the videos have been released! Here are some of the highlights:

If you want the full collection, here's the YouTube playlist.

Alt::BrightonRuby: A Slightly Odd, Quasi-Conference for Strange Times — Alt::BrightonRuby is not happening in-person this year. Instead, you can buy the recorded talks, get a _why book, and get some podcasts with the speakers.

Alt::BrightonRuby

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Security Engineer (Remote) — Are you an engineer with experience in Rails and/or Go? Join our team and help secure our apps and cloud infrastructure.

Shogun

ℹ️ Interested in running a job listing in Ruby Weekly? There's more info here.

???? Articles & Tutorials

▶  How To Begin Contributing to a Gem — If you’ve been using a library for a while and you want to contribute back, how do you get started? A 12 minute introduction here.

Drifting Ruby

How to Set Up Factory Bot on a Fresh Rails Project — Factory Bot is a library for setting up Ruby objects as test data – an alternative to fixtures, essentially.

Jason Swett

Using Postgres's DISTINCT ON to Avoid an N+1 Query“Recently I fixed a tricky N+1 query and thought I should write it up..”

John Nunemaker

Need to Upgrade Rails? Don’t Know How Long It Will Take? — Get an action plan for your Rails upgrade and an in-depth report about your technical debt and outdated dependencies ????.

FastRuby.io | Rails Upgrade Services sponsor

5 Uses for 'Splats' — 5 different ways to leverage Ruby’s splat (*) operator.

Jason Dinsmore

Running Multiple Instances of Webpacker — If you’re working on multiple Rails apps at once, changing where Rails gets served up is easy by configuring the port, but what about Webpacker? That requires another tweak.

Scott Watermasysk

Performing Asynchronous HTTP requests in Rails — How to update parts an app’s pages with asynchronous HTTP requests. A step-by-step how-to with JavaScript’s fetch() function, and Rails native server-side partial rendering.

Remi Mercier

How to Use AWS SimpleDB from Ruby — If you haven’t heard of AWS SimpleDB, you wouldn’t be alone as it’s not very popular, but it’s a pretty simple and cheap way to store simple documents in the cloud.

Peter Cooper

What's The Difference Between Monitoring Webhooks and Background Jobs

AppSignal sponsor

Ways to Reduce Your Heroku App's Slug Size — You might be surprised Heroku didn’t already do some of this for you.

Rohit Kumar

A Chat with Thibaut Barrère — If you missed our interview with Thibaut Barrere (Rubyist, and creator of the Kiba ETL framework) in last week’s issue, you can catch up here.

Glenn Goodrich

???? Code and Tools

Rodauth 2.0: Ruby's 'Most Advanced' Authentication Framework — A authentication framework that can work in any Rack-based webapp. Built using Roda and Sequel, Rodauth can be used with other frameworks and database libraries if you wish. Why’s it so advanced? More info on that here.

Jeremy Evans

RubyGems 3.1.3 Released — Lots of little bug fixes and tweaks.

RubyGems Blog

Business: Business Day Calculations for Ruby — Define your working days and holidays and then you can do ‘business day arithmetic’ (for example, what’s in 5 working days after now taking holidays and weekends into account?)

GoCardless

Lockbox: Modern Encryption for Rails

Andrew Kane

split: The Rack Based A/B 'Split' Testing Framework — A mature framework with robust configuration and multiple options for determining the winning option.

Split

P.S. In last week's issue, one of the links to our sponsors was incorrect and some readers emailed us to say they really wanted to read the promised article, Let’s Explore Big-O Notation with Ruby, so here it is. Apologies for any inconvenience.




an

npm's CTO: So Long, and Thanks for All The Packages

#334 — April 16, 2020

Read on the Web

Node Weekly

npm Has Now (Actually) Joined GitHub — We announced GitHub’s acquisition of npm a month ago but now the process is complete. Not much real news here but the plan is to now focus on community engagement and improving registry infrastructure.

Jeremy Epling (GitHub)

Node v13.13.0 (Current) Releasedfs.readv is a new function to sequentially read from an array of ArrayBufferViews, util.inspect now lets you specify a maximum length for printed strings, the default maximum HTTP header size has been increased to 16KB, there are three new collaborators, and more.

Michaël Zasso

Get Better Insight into Redis with RedisGreen — Modern hosting and monitoring services include memory usage maps, seamless scaling, key size tracking, and more.

RedisGreen sponsor

▶  Watch the Live Coding of a New Feature for Node.js — This is not something for novices, but if the idea of watching ‘over the shoulder’ of a Node.js collaborator implementing a new feature directly into Node itself interests you.. this could be a valuable hour spent.

Vladimir de Turckheim

node-libcurl 2.1: libcurl Bindings for Nodelibcurl is a very powerful and well established way to fetch data from URLs across numerous protocols. node-libcurl 2.1.0 brings support for the latest version of libcurl (7.69.1) to us in the Node world.

Jonathan Cardoso Machado

npm's CTO: 'So Long, and Thanks for All The Packages!' — Ahmad Nassri was npm’s CTO but has now left. Here, he reflects on the past ten years of npm, the repo, the company, and the achievements of both.

Ahmad Nassri

???? Jobs

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

Node.js Developer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

▶️ Get ready for your next role: Pluralsight is free for the entire month of April. Stay Home. Skill Up. #FreeApril — SPONSORED

???? Tutorials

Working With AWS Route 53 from Node — Route 53 is Amazon Web Services’ suite of DNS-related services. Like every AWS service, you can control it via an API, and here’s how to manipulate hosted zones from Node.

Valeri Karpov

Best Practices Learnt Running Express.js in Production for 4 Years — There’s a lot of stuff packed in here focused around middleware, testing, logging, and general concerns around scaling and keeping apps running in production.

Adnan Rahić

The Node.js Security Handbook — Improve the security of your Node.js app with the Node.js security handbook made for developers.

Sqreen sponsor

How To Set Up an Express API Backend Project with PostgreSQL — A pretty extensive walkthrough of creating an HTTP API using Express with Node.js and Postgres on the backend, then deploying it all on Heroku.

Chidi Orji

Porting to TypeScript Solved Our API Woes — From the guy behind the (in)famous Wat video comes a tale of porting a backend from Ruby to TypeScript.

Gary Bernhardt

How to Mass Rename Files in Node

Flavio Copes

▶  Let's Build a Digital Circuit Simulator In JavaScript — A special episode of the Low Level JavaScript series takes us on a brief journey into the world of digital logic.

Low Level JavaScript

The Story of How I Created a Way to Port Windows Apps to Linux — We mentioned ElectronCGI recently as a way to let .NET and Node.js code depend upon each other, but here its creator explains more about the how and why.

Rui Figueiredo

How to Create an Alexa Skill with Node — Implementing a custom ‘skill’ for Amazon Alexa by using Node and AWS Lambda.

Xavier Portilla Edo

???? Tools, Resources and Libraries

Node v10.20.1 (LTS) Released — If you’re still using Node 10, don’t use v10.20.0, use this, due to a bug in the .0 release.

Bethany Nicolle Griggs

emoji-regex: A Regular Expression to Match All Emoji-Only Symbols

Mathias Bynens

ip-num: A Library to Work with ASN, IPv4, and IPv6 Numbers — Happy in both Node and the browser.

dadepo

Optimize Node.js Performance with Distributed Tracing in Datadog

Datadog APM sponsor

verify-json: Verify JSON Using a Lightweight Schema — A lighter weight alternative to something like JSON Schema.

Yusuf Bhabhrawala

middle-manager: A Lightweight 'No BS' Presentation Tool — A bit of humor, really. It turns Markdown into basic presentations but then the magic is it detects your ‘BS’ business language so you can remove it ????

Anders




an

Can you build Node add-ons in Rust? Yes.

#336 — April 30, 2020

Read on the Web

Be sure to check out the Tools and Libraries section today as there have been quite a lot of (minor) releases.. from MIDI parsing and JPEG decoding to generating TypeScript types from a Postgres database.. maybe there's something for you ????

Node Weekly

Middy 1.0: A Node Middleware Framework for AWS Lambda — Middy’s aim is to make writing serverless functions (hosted on AWS Lambda) easier by providing a familiar middleware abstraction to Node developers. The example in this post shows off the main benefit.

Luciano Mammino

Rust and Node.js: A Match Made in Heaven? — This is technical stuff but using other languages, such as Rust, for building add-ons for Node is an interesting area.

Anshul Goyal

Faster CI/CD for All Your Software Projects Using Buildkite — See how Shopify scaled from 300 to 1800 engineers while keeping their build times under 5 minutes.

Buildkite sponsor

Editly: Slick, Declarative Command Line Video Editing — I’ve long wondered why there isn’t a good way to “code” video editing at the command line other than wrangling with arcane ffmpeg options. Well.. this uses ffmpeg, but it handles a lot of the wrangling for you.

Mikael Finstad

Node v14.1.0 (Current) ReleasedLast week we featured the release of Node 14.0 and 14.1 is already with us. Principally bug fixes, plus an update to the OpenSSL dependency.

Bethany Nicolle Griggs

???? Jobs

Backend Developer (Skien, Norway) — We are looking for a full-stack dev with a solid track record to help us adapt to tomorrow's security requirements.

OKAY

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

ℹ️ If you're interested in running a job listing in this newsletter, there's more info here.

???? Articles & Tutorials

Four Tools for Web Scraping in Node — A walk through of a few different libraries (for scraping and parsing data directly from websites) to see how they work and how they compare to each other.

Sam Agnew

Six Platforms for Hosting a Node App in 2020 — Of course, you can run a Node app pretty much anywhere there’s a server, but some platforms make it easier than others. These all have free tiers too. Glitch, Now.sh (now Vercel) and Heroku are particular favorites of ours at Cooperpress.

Amit Bendor

Getting Started with NuxtJS — Learn how to create Vue.js-powered server-side rendered apps with NuxtJS including configuring an app and deploying it on Heroku.

Timi Omoyeni

The Node.js Security Handbook — Improve the security of your Node.js app with the Node.js security handbook made for developers.

Sqreen sponsor

A Collection of Challenging TypeScript Exercises“The goal: Let everyone play with many different TypeScript features and get an overview of TypeScript capabilities and principles.”

Marat Dulin

Exploring Node.js Internals — It’s reasonably elementary but Aleem Isiaka explains how the internals of Node.js interact with one another on a simple task such as creating a file.

Smashing Magazine

Creating CommonJS-Based npm Packages via TypeScript

Dr. Axel Rauschmayer

Turning Vue Components Into Reusable npm Packages — Outlines how you can reuse Vue components across your projects by automating your process to bundle, test, document, and publish your components.

Sjoerd de voorhoede

???? Tools, Resources and Libraries

Node v12.16.3 (LTS) Released — OpenSSL gets an update, and warnings are no longer printed for modules that use conditional exports or package name self resolution.

Node.js

pm2 4.4 Released: The Node Production Process Manager — A very mature and widely used process manager that includes a load balancer for keeping Node apps alive forever and to reload them without downtime. v4.4 improves the Node 14 compatibility.

Alexandre Strzelewicz

jpeg-js: A Pure JavaScript JPEG Encoder and Decoder — It admits it’s far slower than native alternatives but if you need a pure JavaScript JPEG encoder/decoder, this is where to go.

Eugene Ware

AppSignal Now Supports Node.js: Roadmap for the Coming Weeks

AppSignal sponsor

node-stream-zip: For Fast Reading of ZIP Archives — Reads chunk by chunk rather than all in one go so it’s memory friendly.

Dimitri Witkowski

JZZ: A MIDI Library for Node and Web Browsers — Send, receive and play MIDI messages from both Node and the browser on Linux, macOS and Windows.

Sema

Vegemite: A Pub/Sub State Manager — Inspired by Immer and Redux, full TypeScript support, and sized at only 623 bytes, which includes one dependency.

Luke Edwards

Kanel: Generate TypeScript Types from Postgres

Kristian Dupont

web-worker: Consistent Web Workers for the Browser and Node — In Node it works as a web-compatible Worker implementation atop worker_threads. In the browser it’s an alias for Worker.

Jason Miller

node-csv-parse: A CSV Parser Implementing the stream.Transform API

Adaltas




an

An insightful interview with Go's Rob Pike

#310 — May 1, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

An Interview with Go's Rob Pike — Go’s co-creator answers some big picture questions about Go’s status, history, and future. “Go has indeed become the language of cloud infrastructure,” says Rob.

Evrone

???? What's Coming in Go 1.15 — This presentation covers all the major sections: tooling, performance, API changes, and the Big Ones, like the aforementioned smaller binaries. Fingers crossed for a final release in August.

Daniel Martí slidedeck

Troubleshoot Golang App Issues with End-To-End Distributed Tracing — Trace requests across service boundaries to optimize bottlenecks by drilling into individual traces end-to-end with flame graphs. Correlate Golang traces with related logs and metrics for fast troubleshooting. Enhance performance with a free Datadog APM trial.

Datadog APM sponsor

Debugging Go Programs using Delve — The recent Go community survey showed that most Go developers use text-based logging (e.g. with fmt.Print()) to debug, but if you want to step things up a notch, this is a gentle intro to Delve.

Naveen Ramanathan

My Journey Optimizing The Go Compiler — Assel explains how a simple task evolved into a legitimate compiler optimization (aimed at 1.15) and proves we should all have a curious mind.

Assel Meher

The 'Ultimate' Go Study Guide — A large repository of code examples with comments and notes from Hoanh’s attempt at learning the language. If you pick up concepts well from straightforward examples, this is worth a look.

Hoanh An

???? Jobs

Software Engineer at HiPeople (Remote/Berlin) — Fast-moving startup (backed by top tier VCs) shaping the future of modern recruiting is looking for engineers who love working with Go.

HiPeople

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

Making a Multiplayer Game with Go and gRPC — Started as a (somewhat ambitious) project to learn Go, Sam walks us through the algorithms, design decisions, mistakes, and where Go helped and hurt the game.

Samuel Mortenson

Documenting a Go GitHub Repo — Or, “How to Keep the README in Your GitHub Repo in Sync with Your Go Doc.”

Eyal Posener

The 5 Crucial PDF & Office Features For Corporate Apps in Pure Go — UniDoc develops pure Go libraries for managing PDF and Office files since 2016. Here are the features developers use the most.

UniDoc sponsor

▶  Discussing Building Immediate Mode GUIs in Go — Elias Naur, creator of Gio, joins the popular Go podcast to discuss building GUI apps with Go, the pros and cons of immediate vs retained mode and examples of each.

Go Time Podcast podcast

The Creation of a Realtime Patient Monitoring System with Go and Vue in 3 Days — This is the Go content I am here for. Connecting with monitoring devices and leveraging Go’s strengths to create a helpful, distributed application. Great work.

Kasun Vithanage

Add It Up: Azure’s Go Problem — Here’s one takeaway from the Go Developer Survey. Of the major clouds, Azure is the one Go developers seem least enamored by.

Lawrence E Hecht

Why You Should Generally Be using the Latest Version of Go — No surprising arguments here.

Chris Siebenmann

???? Code & Tools

XLSX: A Library for Reading and Writing XLSX (Excel) Files — Got spreadsheets? Want to make spreadsheets? There’s a lot you can do with them here.

Geoffrey J. Teale

SQLBoiler: Generate a Go ORM Tailored to Your Database Schema — A long standing library that has now switched to modules.

Volatile Technologies Inc.

Decimal: Arbitrary-Precision Fixed-Point Decimal Numbers for Go — The library laments that it can only support decimal numbers with up to 2^38 digits after the decimal point so take care ????

Spring Engineering

Beta Launch: Code Performance Profiling - Find & Fix Bottlenecks

Blackfire sponsor

Redigo: A Go Client for Redis — In related news, Redis 6.0 has just been released.

Gary Burd

ntp: Facebook's NTP Libraries — NTP stands for “Network Time Protocol”, if you were wondering. Basically, clock synchronization.

Facebook Incubator

grobotstxt: A Native Go Port of Google's Robots.txt Parser and Matcher Library — Now you can crawl your own site, just like Google does.

Jim Smart

A Compiler for a Small Custom Language Into x86-64 Assembly — One of those ‘labor of love’ type projects that you might enjoy poking around in. You won’t use this project directly, but you might be intrigued how to create a similar compiler for your own thing.

Maurice Tollmien

MIDAS: Microcluster-Based Detector of Anomalies in Edge Streams — A Go reimplementation of this C++ version.

Steve Tan

Liftbridge 1.0: Lightweight, Fault-Tolerant Message Streams — A server that implements a durable, replicated message log for the NATS messaging system.

Liftbridge