Ryft Blog
.avif)
Announcing Ryft Adaptive Optimization
Today, we’re officially introducing Ryft Adaptive Optimization - always-on, dynamic optimization engine for Apache Iceberg™. Our engine continuously compacts, rewrites, indexes, and reorders data based on how your tables are actually used, delivering up to 5× faster queries, 10x storage reduction, and 7x better compaction efficiency compared to other engines.
How to Fix Corrupted Iceberg Tables
In Part 1 and Part 2 of this series, we analyzed two different scenarios that led to Iceberg table corruption - from silent overwrites to inconsistent metadata. Since publishing these posts we have received more requests from people who encountered these situations on how to safely repair those tables. In this post, we’ll focus on the remediation process: identifying what’s affected, how to safely clean it up, and how to prevent further damage.
.avif)
Handling Commit Conflicts in Apache Iceberg: Patterns and Fixes
Commit conflicts in Apache Iceberg are one of those problems that seem rare - until you start operating at scale. The first time a long-running compaction job fails after hours of compute, or a CDC pipeline spends half its time retrying commits, you realize this isn’t a corner case. It’s a core operational challenge that directly impacts cost, latency, and reliability.This post covers what commit conflicts are, why they happen, and how to fix them without creating new problems in the process.
Data Retention in Apache Iceberg: Implementation Details and Best Practices
Data retention in Apache Iceberg is one of those critical operations that seems simple until you implement it at scale. Delete old data, save money, stay compliant - straightforward enough. But the implementation details matter, and getting them wrong can mean failed compliance audits, runaway storage costs, or accidentally purging the wrong data.
.avif)
Announcing Ryft Adaptive Optimization
Today, we’re officially introducing Ryft Adaptive Optimization - always-on, dynamic optimization engine for Apache Iceberg™. Our engine continuously compacts, rewrites, indexes, and reorders data based on how your tables are actually used, delivering up to 5× faster queries, 10x storage reduction, and 7x better compaction efficiency compared to other engines.
Iceberg Table Corruption and Data Loss in the Wild: Part 2
Data and metadata integrity issues in Iceberg, particularly in streaming workloads, often present similar patterns. In these two posts we covered two similar Iceberg table corruption issues, which manifested in almost the exact same way - data file overwrites. Each time, the underlying reason was different.
Why Apache Iceberg Finally Unlocks Security Data Lakes
Security data lakes are notoriously hard to build and operate efficiently. Apache Iceberg changes the equation. It’s an open table format purpose-built for scalable, flexible, and cost-effective analytics - and it’s quickly becoming the new standard for modern cybersecurity data lakes.Here’s why.
.avif)
GDPR Compliance with Apache Iceberg: A Practical Guide
GDPR compliance boils down to one critical requirement: when a user requests deletion of their data, you must delete ALL traces of their “user identifiable information” across ALL systems and copies. Not hide it. Not mark it as deleted. Delete it completely
High performant graph queries on Apache Iceberg powered by Ryft and PuppyGraph
Graph workloads traditionally rely on specialized graph storage systems, but these come with significant challenges in scalability, performance, and data duplication. By combining the storage optimization capabilities of Ryft on Apache Iceberg with PuppyGraph’s advanced graph query engine, teams can run high-performance, scalable graph queries directly on their Iceberg data lake - without moving or duplicating data.
Iceberg Table Corruption and Data Loss in the Wild: Part 1
In this post, we want to share a story about a sneaky bug we encountered that caused table corruption, as well as silent data loss in Iceberg tables. If you're using Iceberg, if your ingestion is based on a streaming pipeline, if you're an AWS EMR user, or if you just like a good bug hunt - read on.
%25201.avif)
Ryft Raises $8M to Help Enterprises Take Control Over Their Data
For years, cloud giants like Snowflake, Databricks, Microsoft, and Google have made billions by offering enterprises an easy way to store and analyze data, as long as that data stays within their platforms. But that convenience came at a hidden cost: soaring expenses, rigid infrastructure, and deep vendor lock-in that slowed innovation and made AI adoption harder. Still, many companies remain stuck in outdated systems, deterred by the complexity of managing their data independently.
Unlocking Iceberg management for everyone
Ryft, in many ways, is a story 15 years in the making. Yuval Yogev, Guy Gadon and I went to the same high school, worked together at 8200, building high-scale data infrastructure, and went our separate ways - all to realize that we really enjoy solving complicated data infrastructure problems together with the people we love.
.avif)
Making Sense of Apache Iceberg Statistics
Apache Iceberg™ is known for its rich metadata model, and one of its most powerful (but often confusing) features is its support for statistics. In this blog post we will break them down, helping you understand what exists today, what you should configure, and what’s coming next.