#300 — April 17, 2020

Read on the Web

Database Weekly

Are Object Stores Starting to Look Like Databases? — Technically, any repository of data could be considered a ‘database’ but now object stores, such as those vast repositories of data sitting behind an S3 API, are beginning to resemble more structured, traditional databases in many ways. This feels a trend and market that will continue to grow in the near future.

Alex Woodie (Datanami)

Event-Reduce: An Algorithm to Optimize Frequently Running Queries — In brief, the idea is that rather than having to re-run queries when data changes on a table, you can basically merge in changes to previous query results. Be sure to check the FAQs.

Daniel Meyer

ACID Transactions in NoSQL? RavenDB Vs MongoDB by Mor Hilai — Where did the stereotype that only relational databases can be fully ACID come from? How did two NoSQL databases, MongoDB & RavenDB, become ACID at the cluster level?

RavenDB sponsor

TerminusDB: A Technical History — We’ve featured it before, but TerminusDB is an open source in-memory graph database built around WOQL (the Web Object Query Language). Here’s an explanation of where it came from and why it exists.

Luke Feeney

Comparing Redis 6's New Multithreaded I/O to ElastiCache and KeyDB — Redis 6 is on the way with threaded I/O being one of the likely new features. KeyDB is a Redis fork whose raison d’etre has been being multithreaded so this comparison may be of interest, though do note that this comes from KeyDB itself.

Ben Schermel (KeyDB)

Intersecting GPS Tracks to Identify Infected Individuals — I’m not a huge fan of COVID-19 related content, but this is a pretty interesting technique with numerous use cases. Essentially it uses PostGIS to identify overlapping paths.

Florian Nadler

Authentication Configuration in PostgreSQL and CockroachDB — In these databases, client authentication can be controlled via a ‘HBA’ (host-based authentication) file.

Raphael ‘kena’ Poss

How MongoDB Enables Machine Learning — If you haven’t played with the popular document-oriented database in a while, you can do quite a few things with it nowadays, including training and using ML algorithms.

Mani Yangkatisal

▶  'We Got that Database', an 'All About that Bass' Parody — This is for fun only! A group of librarians have put together a fun database flavored parody of the rather irritating Meghan Trainor hit ????

Tredyffrin Libraries on YouTube

6 SQL Tricks Every Data Scientist Should Know

Yi Li

Why We Index Everything — Tired of managing indexes to speed up queries? Rockset automatically indexes every field in a row-based store, column-based store, and search index.

Rockset sponsor

Falcon: An Open-Source, Cross Platform SQL Client — Built around Electron and React, this basic client can quickly do chart visualizations of query results and can connect to RedShift, MySQL, PostgreSQL, IBM DB2, Impala, MS SQL, Oracle, SQLite and more.

Plotly

GeoDB: A Persistent Geospatial Database with Geofencing and Google Maps Support — Built using Badger gRPC and the Google Maps API. Track the geolocation of objects across boundaries or in relation to other objects.

Coleman Word

▶️ Get ready for your next role: Pluralsight is free for the entire month of April. Stay Home. Skill Up. #FreeApril — SPONSORED

???? Seen on Twitter..

I think most of us have had this sort of experience with a 'legacy' system before.. ????

#301 — April 24, 2020

Read on the Web

Database Weekly

'Things I Wished More Developers Knew About Databases' — A Google engineer (whose name may be familiar to those Go developers amongst you) shares 17 insights about databases she’s picked up over the years. I strongly recommend this piece and I identify with lots of the points myself..

Jaana B. Dogan

Lambda Store: A New 'Serverless Redis' Service — This seems a neat idea. Claiming to not be just another Redis cloud service, Lambda Store applies a serverless-style pricing model which opens up a variety of neat use cases for the popular data structure server (serverless caching, for starters). The underlying system appears to be a custom clone of Redis rather than the real deal, however.

Sven Anderson

???? AWS, GCP, & Azure Punch Back at the 2020 Cloud Report — AWS, GCP, & Azure each responded to the Cockroach Labs 2020 Cloud Report with instructions on how to tune their respective clouds for optimal performance.

Cockroach Labs sponsor

How io_uring and eBPF Will Revolutionize Programming in Linux — Even more exciting times are coming for development on Linux thanks to these technologies. A good overview from an engineer at ScyllaDB.

Glauber Costa

kvrocks: An Open Source, RocksDB-based, Redis-compatible Database — You know Redis’s API is good when so many projects continue to implement it for themselves. kvrocks brings the Redis API (with pretty good support) together with the RocksDB persistent key-value store. Written in C++.

Bit Leak

Mireo SpaceTime: An Absurdly Fast Spatiotemporal Database? — The SpaceTime database provides unprecedented analytical tools speed, sometimes outperforming other state-of-the-art solutions by three orders of magnitude.

Miljen Mikić

Cloud GPUs Aimed at Data Scientists — Core Scientific, an AI and cloud infrastructure vendor, is teaming with GPU-accelerated analytics specialist SQream Technologies to deliver a “GPU Cloud for Data Scientists.”

Datanami

An Easy Postgres 12 and pgAdmin 4 Setup with Docker — Docker provides an easy and loosely coupled way to get things set up in a development environment.

Jonathan S. Katz

Why We Index Everything — Tired of constantly managing indexes to speed up queries? Learn about how Rockset automatically indexes every field in a row-based store, column-based store, and search index.

Rockset sponsor

Redis Labs Moving RedisJSON to a New Codebase Written in Rust — RedisJSON provides a JSON data type to Redis and it’s been ported from C to Rust for better safety and developer experience.

Gavrie Philipson (Redis Labs)

Replicate Multiple Postgres Servers to a Single MongoDB Server using Logical Decoding Output Plugin

David Zhang

xsv: A Fast CSV Command Line Toolkit Written in Rust — Another ‘Swiss Army knife’ for your slightly structured data.

Andrew Gallant

???? Jobs

DevOps Engineer at X-Team (Remote) — Join the most energizing community for developers. Work from anywhere with the world's leading brands.

X-Team

Data Engineer (Remote - USA Only) — Help us architect and design “big data” systems which require queries returning within sub-second response times.

Social Chorus

#302 — May 1, 2020

Read on the Web

Database Weekly

Redis 6.0 Released — The next major release of the popular data structure server is here. Redis is at the heart of so many data systems nowadays that any major release is big news but 6.0 packs in a lot of new bits and pieces that make it more robust and capable of modern workloads, including:

• Access Control Lists (ACL) for limiting what certain clients can do.
• Diskless replication on replicas.
• Threaded I/O (but Redis itself remains primarily single threaded).
• SSL/TLS support.
• A new client-server protocol, RESP3.

Salvatore Sanfilippo

You Can Now Do Serverless Streaming ETL with AWS Glue — If you want to analyze data on the fly as it arrives, you can now use AWS Glue (AWS’s ETL service) with streaming platforms like Kinesis Data Streams or Kafka which opens up a lot of opportunities as demonstrated here.

Danilo Poccia (AWS)

Monitor Database Performance End-To-End with OOTB Dashboards — Datadog’s customizable, built-in dashboards allow you to collect and visualize custom metrics, like saturation and resource utilization, in real-time. Unify your metrics, traces and logs in one platform. Try it free with a 14-day trial.

Datadog sponsor

Can Apache Kafka Replace a Database? - The 2020 Update — Kafka is a stream-processing system but the idea of using it to store data isn’t uncommon. This post analyzes Kafka’s core concepts from the database perspective.

Kai Waehner

PostgreSQL Gets a Parallel Processing Boost — Swarm64 started life as a FPGA-driven way to accelerate Postgres’s performance for analytics and data warehouse tasks, but can now work without FPGAs too.

Datanami

Using SQL in RStudio

Irene Steves

A Jepsen Security Report on Dgraph 1.1.1 — Jepsen is well known for their work in putting databases and related systems through their security paces. Here, the Go-powered distributed graph database Dgraph gets analyzed.

Jepsen

MongoDB Makes Its Compass GUI Free — Compass provides a powerful interface to work with MongoDB databases. It’s source was made publicly available last year as well.

MongoDB, Inc.

eBook: The Most Important Events to Monitor in Your Postgres Logs — In this eBook, you will learn about the Top 6 Postgres log events for monitoring query performance and preventing downtime.

pganalyze sponsor

Pantry: Free Cloud-Based Storage for JSON Data — Provides “perishable”, though secure, data storage for small projects via a RESTful API. Data is erased after a period of inactivity. There are quite a few projects like this and while they don’t suit long term production use, they can come in handy for hackathons, quick personal projects, teaching, etc.

Rohan Likhite

Liftbridge 1.0: Lightweight, Fault-Tolerant Message Streams — A server that implements a durable, replicated message log for the NATS messaging system.

Liftbridge

#303 — May 8, 2020

Read on the Web

Database Weekly

Introducing Scylla Open Source 4.0 — Scylla (a Cassandra-compatible NoSQL data store aiming to be the “world’s fastest column-store database”) now provides production-ready lightweight Transactions (LWT), a DynamoDB-compatible API (Alternator), and operator for Kubernetes, and more.

Dor Laor

The Best Medium-Hard Data Analyst SQL Interview Questions — This article begins with a quote: “The first 70% of SQL is pretty straightforward but the remaining 30% can be pretty tricky.” True! This article focuses on the tricky ‘medium-hard’ area that few tutorials venture into.

Zachary Thomas

????Live Coding: Guide to Grafana 101 - Getting Started with Alerts — Join us on May 20th to see how to use Grafana’s alerting functionality to get notified about anomalies in your data, dig into root causes, and respond to critical issues. Step-by-step demos + tips = cheaper, more flexible monitoring ✅.

Timescale sponsor

TileDB 2.0 and the Future of Data Science — TileDB is an embeddable storage engine focused on working with dense and sparse multi-dimensional arrays. It’s a C++ library with official Python, R, Java and Go integrations, but it can integrate with other database systems too. 2.0 introduces dataframe support, a new API for R, and support for Google Cloud Storage and Azure Blob Storage.

Stavros Papadopoulos

Time-Series Compression Algorithms, Explained — Delta-delta encoding, Simple-8b, XOR-based compression, and more - these algorithms aren’t magic, but combined they can save over 90% of storage costs and speed up queries. Here’s how they work.

Joshua Lockerman and Ajay Kulkarni

CockroachDB Creators Raise $87 Million of New Investment — Quite a raise and quite a valuation in these times for the creators of CockroachDB, a popular distributed SQL database.

Cockroach Labs

The Big Cloud Data Boom Gets Even Bigger, Thanks to COVID-19? — It’s not like the cloud was doing badly beforehand, but the pandemic is apparently encouraging companies to virtualize as much of their operations as possible.

Datanami

MongoDB Is Easy. Now Make It Powerful. Free Download for 30 Days. — Using MongoDB Atlas? Studio 3T is the professional GUI and IDE that unlocks the power you need.

Studio 3T sponsor

Speeding Up count(*): Why Not Use max(id) - min(id)? — A warning tale in case you decide to take this shortcut. While you might be able to estimate or fudge a number that’s close, you can’t guarantee sequences will give you an exact, correct answer here.

Hans-Jürgen Schönig

Using AWS API Gateway to Run Database Queries — API Gateway is commonly used to hook up HTTP endpoints to AWS Lambda functions but did you know it can directly connect to DynamoDB? (Or any AWS service that lets you query over the AWS API, so not RDS.)

Renato Byrro

How to Remain Agile with DynamoDB — Amazon DynamoDB delivers performance at scale but at a cost to flexibility (particularly early on in the development cycle when your eventual access patterns aren’t always known) – there are some mitigations, however.

Rob Cronin

Jobs

DevOps Engineer at X-Team (Remote) — Join X-Team and work on projects for companies like Riot Games, FOX, Coinbase, and more. Work from anywhere.

X-Team

Tooling

pgModeler: A Postgres Database Modeler — An easy way to create and edit database models in a visual way. It’s packaged up as a paid product but is also open source so you can build your own.

Raphael Araújo e Silva

AvionDB: A Decentralised Database with MongoDB-like Developer Interface — An admittedloy ‘alpha stage’ database system built on top of OrbitDB, a serverless, peer-to-peer database that uses IFPS for storage and implements the core decentralized database logic/protocol.

Dappkit

mssql-cli, a CLI Tool to Manage SQL Server, Now on macOS and Linux — mssql-cli is a tool for working with SQL Server from the command line, complete with Intellisense, syntax highlighting, and paging.

Alan Yu (Microsoft)