#302 — May 1, 2020

Read on the Web

Database Weekly

Redis 6.0 Released — The next major release of the popular data structure server is here. Redis is at the heart of so many data systems nowadays that any major release is big news but 6.0 packs in a lot of new bits and pieces that make it more robust and capable of modern workloads, including:

• Access Control Lists (ACL) for limiting what certain clients can do.
• Diskless replication on replicas.
• Threaded I/O (but Redis itself remains primarily single threaded).
• SSL/TLS support.
• A new client-server protocol, RESP3.

Salvatore Sanfilippo

You Can Now Do Serverless Streaming ETL with AWS Glue — If you want to analyze data on the fly as it arrives, you can now use AWS Glue (AWS’s ETL service) with streaming platforms like Kinesis Data Streams or Kafka which opens up a lot of opportunities as demonstrated here.

Danilo Poccia (AWS)

Monitor Database Performance End-To-End with OOTB Dashboards — Datadog’s customizable, built-in dashboards allow you to collect and visualize custom metrics, like saturation and resource utilization, in real-time. Unify your metrics, traces and logs in one platform. Try it free with a 14-day trial.

Datadog sponsor

Can Apache Kafka Replace a Database? - The 2020 Update — Kafka is a stream-processing system but the idea of using it to store data isn’t uncommon. This post analyzes Kafka’s core concepts from the database perspective.

Kai Waehner

PostgreSQL Gets a Parallel Processing Boost — Swarm64 started life as a FPGA-driven way to accelerate Postgres’s performance for analytics and data warehouse tasks, but can now work without FPGAs too.

Datanami

Using SQL in RStudio

Irene Steves

A Jepsen Security Report on Dgraph 1.1.1 — Jepsen is well known for their work in putting databases and related systems through their security paces. Here, the Go-powered distributed graph database Dgraph gets analyzed.

Jepsen

MongoDB Makes Its Compass GUI Free — Compass provides a powerful interface to work with MongoDB databases. It’s source was made publicly available last year as well.

MongoDB, Inc.

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, you will learn about the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Pantry: Free Cloud-Based Storage for JSON Data — Provides “perishable”, though secure, data storage for small projects via a RESTful API. Data is erased after a period of inactivity. There are quite a few projects like this and while they don’t suit long term production use, they can come in handy for hackathons, quick personal projects, teaching, etc.

Rohan Likhite

Liftbridge 1.0: Lightweight, Fault-Tolerant Message Streams — A server that implements a durable, replicated message log for the NATS messaging system.

Liftbridge

#303 — May 8, 2020

Read on the Web

Database Weekly

Introducing Scylla Open Source 4.0 — Scylla (a Cassandra-compatible NoSQL data store aiming to be the “world’s fastest column-store database”) now provides production-ready lightweight Transactions (LWT), a DynamoDB-compatible API (Alternator), and operator for Kubernetes, and more.

Dor Laor

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

????Live Coding: Guide to Grafana 101 - Getting Started with Alerts — Join us on May 20th to see how to use Grafana’s alerting functionality to get notified about anomalies in your data, dig into root causes, and respond to critical issues. Step-by-step demos + tips = cheaper, more flexible monitoring ✅.

Timescale sponsor

TileDB 2.0 and the Future of Data Science — TileDB is an embeddable storage engine focused on working with dense and sparse multi-dimensional arrays. It’s a C++ library with official Python, R, Java and Go integrations, but it can integrate with other database systems too. 2.0 introduces dataframe support, a new API for R, and support for Google Cloud Storage and Azure Blob Storage.

Stavros Papadopoulos

Time-Series Compression Algorithms, Explained — Delta-delta encoding, Simple-8b, XOR-based compression, and more - these algorithms aren’t magic, but combined they can save over 90% of storage costs and speed up queries. Here’s how they work.

Joshua Lockerman and Ajay Kulkarni

CockroachDB Creators Raise $87 Million of New Investment — Quite a raise and quite a valuation in these times for the creators of CockroachDB, a popular distributed SQL database.

Cockroach Labs

The Big Cloud Data Boom Gets Even Bigger, Thanks to COVID-19? — It’s not like the cloud was doing badly beforehand, but the pandemic is apparently encouraging companies to virtualize as much of their operations as possible.

Datanami

MongoDB Is Easy. Now Make It Powerful. Free Download for 30 Days. — Using MongoDB Atlas? Studio 3T is the professional GUI and IDE that unlocks the power you need.

Studio 3T sponsor

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using AWS API Gateway to Run Database Queries — API Gateway is commonly used to hook up HTTP endpoints to AWS Lambda functions but did you know it can directly connect to DynamoDB? (Or any AWS service that lets you query over the AWS API, so not RDS.)

Renato Byrro

How to Remain Agile with DynamoDB — Amazon DynamoDB delivers performance at scale but at a cost to flexibility (particularly early on in the development cycle when your eventual access patterns aren’t always known) – there are some mitigations, however.

Rob Cronin

Jobs

DevOps Engineer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Tooling

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

AvionDB: A Decentralised Database with MongoDB-like Developer Interface — An admittedloy ‘alpha stage’ database system built on top of OrbitDB, a serverless, peer-to-peer database that uses IFPS for storage and implements the core decentralized database logic/protocol.

Dappkit

mssql-cli, a CLI Tool to Manage SQL Server, Now on macOS and Linux — mssql-cli is a tool for working with SQL Server from the command line, complete with Intellisense, syntax highlighting, and paging.

Alan Yu (Microsoft)

#352 — April 22, 2020

Read on the Web

???? We've got a neat bonus for you this week at the bottom of the issue ????

Postgres Weekly

Postgres Explain Visualizer 2: A Vue.js Component to Show Execution Plans — Less a standalone tool and something you’d use when building your own Postgres tooling. There is, however, a demo here. The output is really nifty.

Dalibo

Insert-Only Tables To Be Autovacuumed in Postgres 13 (But Why?) — Autovacuuming clears up dead tuples that are often left when updating or deleting data from tables, so why is autovacuuming for append-only tables a big deal in Postgres 13? Laurenz explains.

Laurenz Albe

Faster CI/CD for All Your Software Projects Using Buildkite — See how Shopify scaled from 300 to 1800 engineers while keeping their build times under 5 minutes.

Buildkite sponsor

Full Text Search in Milliseconds with Rails and Postgres — If you’ve never played with full text search with Postgres and Rails, this is a fine place to start. It covers LIKE/ILIKE, trigrams, and ‘proper’ full text searching. We also get to see how Leigh took a query from taking 130ms down to 7ms.

Leigh Halliday

An Easy Postgres 12 and pgAdmin 4 Setup with Docker — Docker provides an easy and loosely coupled way to get things set up in a development environment.

Jonathan S. Katz

Is There a Limit on Number of Partitions Handled by Postgres? — Sort of, but you’d really have to be going at it to stretch Postgres 12’s capabilities in this area.

Denish Patel

Where Do My Postgres Settings Come From? — A nice visual look at how parameters and settings cascade or override each other.

My DBA Notebook

Identify Slow-Running PostgreSQL Queries Quickly in Datadog — Improve PostgreSQL performance by visualizing and identifying errors fast using granular, out-of-the-box dashboards in Datadog.

Datadog sponsor

Replicate Multiple Postgres Servers to a Single MongoDB Server using Logical Decoding Output Plugin

David Zhang

An Overview of the JOIN Methods in Postgres

Kumar Rajeev Rastogi

If you enjoyed this interview, Luca actually gave some more detailed answers in the full interview which you can read here.

#354 — May 6, 2020

Read on the Web

Postgres Weekly

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

Monitor Custom Postgres Metrics in Real-Time with Datadog — Monitor and visualize Postgres performance in context end-to-end alongside the rest of your stack. Create custom, drag-and-drop dashboards to quickly view analytics on any Postgres metric. Try Datadog free.

Datadog sponsor

My Favorite PostgreSQL Extensions: Part Two — The second part of a series we linked to last week. This time, Nawaz takes a look at pgAudit, pg_repack, and HypoPG.

Nawaz Ahmed

Backup Manifests and pg_verifybackup in Postgres 13 — Postgres 13 will introduce two features to enhance the automated validation of physical backups: backup manifests and a new tool called pg_verifybackup.

Gabriele Bartolini

arm64 Packages Now on apt.postgresql.org — If you’re running ARM64 hardware and Debian or Ubuntu, you can now install Postgres via apt.

Christoph Berg

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using Postgres for JSON Storage — With JSON and JSONB types and associated advanced ways to query such columns, using Postgres as a store for JSON data is pretty simple. This is the briefest of overviews but leads into an interactive online tutorial.

Steve Pousty

How to Migrate From Inheritance-Based Partitioning to Declarative Partitioning — Partitioning was introduced in Postgres 10 and Postgres 11 improved the declarative partitioning support. This article demonstrates a move from inheritance based partitioning to declarative partitioning using the native features found in Postgres 11+.

Caterina Magini

Free eBook: How to Get a 3x Performance Improvement on Your Postgres Database — Learn our best practices for optimizing Postgres query performance for customers like Atlassian and how to reduce data loaded from disk by 500x.

pganalyze sponsor

How to Backup Multiple Tablespaces with pg_basebackup

Ahsan Hadi

▶  Security and Compliance with Postgres — A recorded webinar that 2ndQuadrant ran recently.

Boriss Mejías

Oracle to PostgreSQL: ANSI Outer Join Syntax in Postgres — The latest in a series of blog posts about migrating to Postgres from Oracle which looks at what Postgres offers in place of Oracle’s join operators.

Kirk Roybal

An Interview with 2ndQuadrant's Jimmy Angelakos — The latest ‘PostgreSQL Person of the Week’ to face questions about his experiences with Postgres.

Andreas Scherbaum

dadbod.vim: A Modern Database Interface for Vim — A Vim plugin for interacting with numerous databases, including Postgres.

Tim Pope

#335 — April 23, 2020

Read on the Web

Node Weekly

Node.js 14 Released — Woo-hoo another major release of Node.js is here. v14 now becomes the current ‘release’ line with it becoming a LTS (Long Term Support) release in October.. so production apps would, ideally, remain on v12 for now. So what’s new..?

• Diagnostic reports are now a stable feature.
• It's based on V8 8.1.
• An experimental Async Local Storage API
• Improvements to streams.
• An experimental WebAssembly System Interface (WASI) to support future WebAssembly use cases.
• Bye bye to the ESM module ‘experimental’ warning (though it still is experimental).

Michael Dawson and Bethany Griggs

Learn Hardcore Functional Programming in JavaScript — Join Brian Lonsdorf and learn how to apply such concepts as pure functions, currying, composition, functors, monads and more.

Frontend Masters sponsor

Puppeteer 3.0: Say Hello to Firefox — Best known as a way to headlessly control Chrome from Node, Puppeteer has recently seen some competition in the form of the cross-browser Playwright recently. But competition can be good and Puppeteer now supports Firefox too.

Mathias Bynens

ZEIT Is Now Vercel — You probably best know ZEIT as the creators and maintainers of the popular Next.js React framework and their ‘Now’ deployment and hosting platform.

Vercel

New OpenSSL Security Release To Require Node Updates? Maybe Not.. — A key security update to OpenSSL raised the possibility of widespread Node releases to incorporate the fixes, but initial suggestions are that Node isn’t affected. Fingers crossed!

Sam Roberts

???? Jobs

Node.js Developer at X-Team (Remote) — Join the most energizing community for developers. Work from anywhere with the world's leading brands.

X-Team

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

OneTesselAway: Building a Real-Time Public Transit Status Device — A developer wanted to know when the next bus would arrive.. while using lots of cool tech, including Node, the OneBusAway API, and the Tessel 2 IoT platform.

Robert McGuire

What is the toJSON() Function? — If an object has a toJSON function, JSON.stringify() calls toJSON() and serializes the return value from toJSON() instead.

Valeri Karpov

Top GitHub Best Practices for Developers - Expanded Guide — Implementing these best practices could save you time, improve code maintainability, and prevent security risks.

Datree.io sponsor

Refactor Your Node and Express APIs to Serverless with Azure Functions — John Papa points out a Microsoft tutorial that walks through the process of taking a Node API serverless with Azure’s Functions service.

John Papa

Querying SQL Server from Node with async/await

Rob Tomlin

Why I Stopped Using Microservices

Robin Wieruch

???? Tools, Resources and Libraries

lazynpm: A Terminal UI for npm — One of those sort of things you don’t realize you need until you give it a go. There’s a four-minute screencast if you want to see how it works without downloading.

Jesse Duffield

node-sqlite3 4.2: Async, Non-blocking SQLite3 Bindings for Node — 4.2.0 just came out.

Mapbox

ts-gphoto2-driver: A Node Wrapper for libgphoto2 — libgphoto2 provides a way to control a variety of digital cameras/DSLRs.

Lenzotti Romain

AppSignal Now Supports Node.js: Roadmap for the Coming Weeks

AppSignal sponsor

Rosetta: A General Purpose Internationalization Library in 292 Bytes — Less than 300 bytes, but does have a few dependencies. Aims to be very simple and is targeted at basic string use cases.

Luke Edwards

nodejs-dns 2.0: The Google Cloud DNS Client for Node

Google

node-osc 5.0: Open Sound Control Protocol Library — OSC is a protocol used to communicate between media devices.

Myles Borins

ts-node: TypeScript Execution and REPL for Node

TypeStrong

???? And One for Fun..

npm trends: Compare NPM Package Downloads — A site to compare package download counts over time. For example, what about koa vs restify vs fastify?

John Potter

#311 — May 8, 2020

Unsubscribe  :  Read on the Web

Golang Weekly

Caddy 2: The Go-Powered Web Server with Automatic TLS — After over a year of redesign, Caddy 2 has a new architecture to v1. If you want a new HTTPS server that ‘just works’, Caddy is well worth a look IMO. Its lead creator, Matt Holt, answered lots of questions on this Hacker News thread about the release.

Caddy Web Server

Rek: An Easy HTTP Client for Go — The inspiration here is from Python’s very well known and highly esteemed Requests library.. so the Pythonistas among you might like this!

Luc Perkins

Modern Redis Features with RedisGreen — Online upgrades to the latest Redis 6.0 features, memory mapping, key size tracking, and more.

RedisGreen sponsor

Life Without Line Numbers — There’s a lot of buzz around reducing the size of Go binaries (1.15 does so by ~6%) and here’s another tactic: reduce the precision of the position information. The gain is 2-6%, depending on how far you take it.

Josh Bleecher Snyder

▶  Discussing Black Hat Go — “Are you excited to learn about hacking and that?” Got an hour? Roberto Clapis, a security engineer at Google, and Tom Steele, a co-author of Black Hat Go, join the Go Time team to discuss security, penetration testing, and more.

Go Time Podcast

???? Jobs

Enjoy Building Scalable Infrastructure in Go? Stream Is Hiring — Like coding in Go? We do too. Stream is hiring in Amsterdam. Apply now.

Stream

Find a Job Through Vettery — Vettery specializes in tech roles and is completely free for job seekers. Create a profile to get started.

Vettery

???? Articles & Tutorials

Mid-Stack Inlining in Go — Inlining a function can lead to serious performance gains, so why not do it for everything? Well, there are always trade-offs.

Dave Cheney

Asynchronous Preemption in Go 1.14 — How the new preemption implementation works, including the use of a lesser-known signal (SIGURG).

Vincent Blanchon

Why Are My Go Executable Files Larger Than My Source Code? — We built a data visualization tool to find out. Here’s how we built it, and what we learned.

Cockroach Labs sponsor

Accelerating Aggregate MD5 Hashing Up to 800% with AVX512 — The culmination of this work is md5-simd, a Go library that performs such rapid MD5 hashing (when running concurrently). The use cases here are quite restricted but you may appreciate seeing how such things are implemented for any high end SIMD wrangling you need to do one day.

MinIO Blog

▶  A Beginner's Guide to gRPC in Go — There’s a written version of the tutorial if you dislike videos.

TutorialEdge

Four Steps to Daemonize Your Go Programs — Daemons are programs that run as non-interactive background processes (e.g. background job processors, Web servers, database systems).

Ilija Eftimov

Go as a Scripting Language? — There’s plenty of folks that use Go as a scripting language, but there are challenges around REPLs and shebang support. Some of these challenges are being addressed today.

Segio De Simone

???? Code & Tools

UUID 3.3: A Pure Go Implementation of UUIDs — A pure Go implementation of Universally Unique Identifiers (UUID) as defined in RFC-4122 covering versions 1 through 5.

The Go Commune

Reed-Solomon: A Reed-Solomon Erasure Coding Library — A Go port of a Java library built by Backblaze that does Reed Solomon erasure coding (a way to send or store data in a larger form that’s resilient to data loss). Boasts operation of over 1GB/sec per core.

Klaus Post

ko 0.5: Build and Deploy Go Apps on Kubernetes — ko’s objective is to “to make containers invisible infrastructure.” It’s been rapidly maturing in the past few months too.

Google

Monitor the Health and Performance of Your Golang Apps with Datadog APM. Free Trial

Datadog APM sponsor

Tengo 2.2: A Fast Embeddable Script Language for Go — Quite a mature project now and worth a look if you need to add some dynamic scripting to your code.

Daniel Kang

UniPDF 3.7: A Library for Creating and Processing PDF Files — Pure Go, which is neat, but note it’s dual licensed: AGPL for open source, commercial for closed source projects.

UniDoc

Mockery: A Mock Code Generator for Go Interfaces

Vektra

Dynamo: An Expressive DynamoDB Library

Greg Greg

???? Two Fun Side Projects

gasm: An Experimental WASM Virtual Machine for Gophers — “I did this just for fun and for learning WASM specification.” Nonetheless, it works with basic examples.

Takeshi Yoneda

thdwb: A Homebrew Web Browser and Rendering Engine — Another experimental, fun learning project. You won’t be using it for your day to day browsing any time soon but projects like this keep the imagination fueled up.

Danilo Fragoso

It'd be quite cool to link to more fun Go experiments and side projects actually, so let us know if you work on any. Bonus points for games, musical, or Web experiences ????