The End of Data Copies? What a Google Cloud Iceberg Demo Reveals About the Future of Analytics
For years, the modern data stack has suffered from an oddly accepted inefficiency: everybody gets their own copy of the data.
To illustrate an example of it: Analytics teams using BigQuery. Data engineers building pipelines in Spark. Platform teams deploying Trino for federated and ad hoc querying. To make it all work together, organizations create synchronization pipelines, maintain duplicate storage locations, and hope those copies stay aligned.
The result is familiar.
One system says an order has shipped. Another says it is still pending. Someone launches an investigation, discovers a failed sync job, and several hours disappear into operational troubleshooting.
However this KDNuggets article gives a demonstration of Apache Iceberg highlighting it as a compelling alternative: one table, shared by multiple engines, without synchronization jobs or duplicated datasets.
The Promise of "One Table, Many Engines"
The architecture demonstrated in the article is deceptively simple.

Instead of maintaining separate copies of data for BigQuery, Spark, and Trino, all three engines interact with a single Apache Iceberg table stored in Cloud Storage. The engines share not only the same Parquet files but also the same metadata definition and table history.
The implications are significant:
- No ETL processes solely for synchronization
- No duplicate storage costs
- Fewer consistency problems
- Immediate visibility of changes across engines
- Reduced operational overhead
In the demo, Spark creates the table and loads data, Trino appends new records, and BigQuery performs updates and merges. Each engine sees the changes made by the others on subsequent queries without any synchronization pipeline running in the background.
For organizations pursuing a lakehouse architecture, this is remarkably close to the idealized vision the industry has discussed for years.
The Real Innovation Isn't Storage
The most interesting part of the article is not that multiple engines can read the same files.
We have been able to do that for quite some time.
The truly important capability is coordinated writes.
Multiple engines making concurrent changes to the same dataset is where many architectures become fragile. Data corruption, lost updates, race conditions, and inconsistent views of data have historically forced organizations to create rigid ownership boundaries.
Apache Iceberg addresses this through snapshot-based metadata management and optimistic concurrency control. Rather than directly overwriting data, writers create new metadata versions and attempt to commit them. If another writer has already modified the table, the competing transaction must reconcile with the latest state before proceeding.
That mechanism provides something data teams desperately need: Consistency without centralization.
What Happens When Two Engines Collide?
The most valuable section of the demonstration intentionally creates a conflict.
A long-running Spark MERGE operation is launched, and before it can complete, Trino inserts new records into the same table. The goal is to see how Iceberg behaves when two engines attempt overlapping modifications.
The result is reassuring.
Under serializable isolation, Trino commits successfully. Spark detects that the table changed while its transaction was in progress and aborts rather than risking inconsistent results. No corruption occurs. No data is lost. Readers continue seeing complete snapshots throughout the process.
This behavior highlights an important principle:
A modern lakehouse is not merely a collection of files. It is a transactional system.
The catalog and metadata layer have become just as important as the storage layer itself.
Isolation Levels Matter More Than Most Teams Realize
The article also demonstrates a second scenario using snapshot isolation.
After changing the table's merge isolation level, both the Spark merge and Trino append are able to complete successfully. The tradeoff is that newly inserted rows are not included in the already-running merge operation.
This exposes a broader lesson for data architects.
There is no universally correct isolation strategy.
- Financial balances may require strict serializable guarantees.
- Inventory systems may prioritize correctness over throughput.
- Event-driven pipelines may accept snapshot semantics in exchange for higher concurrency.
- Operational analytics often benefit from allowing multiple writers to coexist.
Choosing isolation levels is increasingly becoming an architectural decision rather than merely a database configuration setting.
The Hidden Story: Data Platforms Are Becoming Interchangeable
Perhaps the biggest takeaway is not about Iceberg itself.
It's about what happens when storage and table formats become the primary abstraction.
Historically, choosing a query engine often meant choosing where data lived and how it was managed.
That assumption is weakening.
When BigQuery, Spark, Trino, DuckDB, Snowflake, and other systems can operate against common open table formats, the competitive landscape changes. Organizations become less dependent on a single processing engine and gain flexibility to select the right tool for each workload.
The data platform stops being a monolith.
Instead, it becomes a set of interchangeable compute engines operating on shared data assets.
That may be one of the most important shifts occurring in data engineering today.
The Remaining Caveats
The demonstration is impressive, but it is not without limitations.
The author points out several practical considerations:
- Long-running writes can repeatedly lose conflicts against fast append workloads.
- Merge operations should be scoped narrowly whenever possible.
- Retry settings matter significantly in high-concurrency environments.
- Some engines still have feature gaps or compatibility limitations.
- Certain write patterns may still require separate data copies.
In other words, the need for architecture does not disappear simply because the synchronization jobs do.
Final Thoughts
Apache Iceberg's value proposition isn't that it stores data differently.
Its real value is that it allows organizations to treat data as a shared asset instead of a collection of synchronized copies.
The Google Cloud demonstration shows that the vision is no longer theoretical. BigQuery, Spark, and Trino can operate on the same table, commit transactions safely, and provide consistent views of data without constant synchronization overhead.
For data teams exhausted by replication pipelines and reconciliation exercises, that may be the most compelling promise of the lakehouse movement yet: One table. One source of truth. Multiple engines.
Author: David G. Smith • Source: https://davidgsmith.net/thoughts/the-end-of-data-copies-what-a-google-cloud-iceberg-demo-reveals-about-the-future-of-analytics.html
Related Thoughts
Perspectives sharing related architectures, models, and domain context.
DuckDB 2.0.1: The Release That Shows DuckDB Is Growing Up
Most software projects become popular because of new features. The successful ones become boring. At first glance, the...
CNG Forum 2026 Sessions Posted to YouTube
If like me you missed the Cloud-Native Geospatial (CNG) Forum 2026 in Snowbird, Utah this October, or if you were there...
Cloud-Native Map Tiles With No Drama
For a long time, running your own map service was a real chore. You didn't just need map data. You needed tile servers....