Project Metamorphosis: Unveiling the next-gen event streaming platformLearn More
July | Infinite

Infinite retention for Apache Kafka® with Confluent

  • Transform Kafka into a central nervous system for all events
  • Infinitely retain events without pre-provisioning storage
  • Reduce storage costs up to 70% in Confluent Platform

Infinite resources

July 1

Infinite Storage in Confluent Cloud for Apache Kafka

Learn what unlimited storage unlocks in Confluent Cloud

Aug 4

Infinite Retention in Confluent Cloud

See how to set up infinite storage and retention in Confluent Cloud

Why infinite matters

Transform Kafka into your central nervous system for all events

In traditional data architectures, storage systems focus on accumulating past records while messaging services are waiting to process future events. On top of this data layer, there may be hundreds of in-house systems, SaaS applications, or microservices both generating and consuming data. Data integrity and correctness must be maintained across all these systems, which is a huge operational burden and business risk as organizations and apps must make quick decisions on what they believe is the most accurate and up to date version of the data while retaining what is needed for regulatory compliance. Confluent's event streaming solutions allow you to:

  • With no limits on storage, retain all historical and current events in Kafka as long as you want
    • Set retention at the topic level to be as long as you want without scaling additional compute resources to handle the storage
    • Reduce tech debt and downtime risk with a central source of truth for all events across your entire ecosystem
  • Democratize access to your data with all events in one place and eliminate the need for complex data engineering
    • Maintain data compatibility with Schema Registry and enforce data integrity with Schema Validation to build an infinite and immutable central source of truth for all streaming events
    • Keep the flexibility to use your programming language of choice and transform infinite amounts of data in Kafka using clients such as C, .NET, Java, GO, Python and more
  • Reduce business risk by setting retention policies that help maintain regulatory compliance and providing a persistent log of all events for compliance audits
Infinitely retain events without ever worrying about storage

Provisioning massive Kafka clusters and retaining the right amount of data in the cloud with Apache Kafka is complex. Setting it up properly to handle GBps throughputs and petabytes of data can turn into months of work that push delivery timelines delaying business go-to-market. Similarly, businesses that don't need GBps scale or petabytes of storage at the beginning but face unexpected spikes or storage needs risk causing downtime or degrading cluster performance, which can lead to customer frustration. Confluent Cloud allows you to:

  • Never worry about Kafka storage again in Confluent Cloud with elastically scalable storage that automatically grows with your traffic
    • Avoid downtime and data loss by auto-scaling storage based on retention policies and traffic
  • Only pay for storage used with Confluent Cloud instead of paying for over provisioned storage volumes attached to Kafka brokers
    • Only pay for retained data, not pre-provisioned storage capacity
    • Decouple storage and compute to avoid unnecessary compute costs for high storage use cases
Reduce storage costs up to 70% with tiered storage

Storing historical data on self-managed Kafka clusters quickly increases your infrastructure costs because you have to over provision storage to prevent downtime and deploy larger clusters to avoid hitting storage limitations on brokers. It becomes cost-prohibitive to retain all the data you need in Kafka, causing organizations to glue together disjointed architectures that end up costing more money and siphoning off more engineering resources. Confluent Platform allows you to:

  • Send long term storage to cost-efficient object storage, such as S3 and GCS, using Tiered Storage in Confluent Platform
    • Expand cluster storage in Confluent Platform without any additional resource provisioning when you offload data to 3rd party object storage
  • Reduce partition rebalancing time for self-managed clusters from hours to seconds using Tiered Storage to minimize the size of your partitions on brokers

利点

Scalable Storage
無制限の保存
データの互換性
Confluent Cloud

Decouple compute and storage in Kafka to elastically scale storage

Infinitely retain events in Kafka only paying for storage used

Enforce consistent data formats for app development

Confluent Platform

Decouple compute and storage in Kafka to elastically scale storage

Infinitely retain events in Kafka by offloading data to object storage

Programmatically enforce data schemas at the broker-level

機能

  • July

    Infinite retention

    Store data in Basic and Standard clusters on AWS for as long as it is needed

  • Available

    Schema Registry

    Enforce consistent data formats to enable app development compatibility with Avro, Protobuf & JSON

  • July

    Schema validation

    Enable centralized policy enforcement and data governance within your event streaming platform

  • Available

    Multi-language development

    Enable all developers to leverage Kafka throughout the organization

  • プレビュー

    Tiered storage

    Retain infinite data in Kafka by offloading older topic data to cost-effective object storage

  • Available

    Schema Registry

    Enforce consistent data formats to enable app development compatibility with Avro, Protobuf & JSON

  • Available

    Schema validation for Schema Registry

    Programmatically validate and enforce schemas at the broker-level

  • Available

    Multi-language development

    Enable all developers to leverage Kafka throughout the organization

More Project Metamorphosis releases

Elastic Scaling

How do you quickly scale Kafka to keep mission-critical apps running with no lag or downtime - and without over-provisioning expensive resources?

詳細を読む

Cost Effective

How do you keep the costs of running Kafka low and your best people focused on critical projects driving competitive advantage and revenue?

詳細を読む

Everywhere

How can you ensure your Kafka infrastructure is flexible enough to adapt to your changing cloud requirements?

詳細を読む

Try it out

クラウド
完全マネージドサービス

わずか数分でデプロイ。お支払いは利用された分だけ。サーバーレスの Kafka 体験をお試しください。

プラットフォーム
自己管理型ソフトウェア

無料ダウンロードでエンタープライズ対応プラットフォームのパワーを体験できます。

*Receive $200 off your bill each calendar month for the first three months

Sign Up Now

Start your 3-month trial. Get up to $200 off on each of your first 3 Confluent Cloud monthly bills

新規登録のみ。

上の「新規登録」をクリックすることにより、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします : プライバシーポリシー

上記の「新規登録」をクリックすることにより、お客様は以下に同意するものとします。 サービス利用規約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします: プライバシーポリシー

Get Confluent Cloud

Get up to $200 off on each of your first 3 Confluent Cloud monthly bills


Choose one sign-up option below

Marketplaces

  • AWS
  • Azure
  • Google Cloud

  • Billed through your Cloud provider*
  • Stream only on 1 cloud
*Billing admin role needed

Marketplaces

  • Billed through your Cloud provider*
  • Stream only on 1 cloud
  • Billing admin role needed

*Billing admin role needed

Confluent


  • Pay with a credit card
  • Stream across multiple clouds

Confluent

  • Pay with a credit card
  • Stream across multiple clouds

上の「新規登録」をクリックすることにより、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします : プライバシーポリシー

上記の「新規登録」をクリックすることにより、お客様は以下に同意するものとします。 サービス利用規約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします: プライバシーポリシー

単一の Kafka Broker の場合には永遠に無料
i

商用版の機能を単一の Kafka Broker で無期限で使用できるソフトウェアです。2番目の Broker を追加すると、30日間の商用版試用期間が自動で開始します。この制限を単一の Broker へ戻すことでリセットすることはできません。

デプロイのタイプを選択
手動デプロイ
  • tar
  • zip
  • deb
  • rpm
  • docker
または
自動デプロイ
  • kubernetes
  • ansible

上の「無料ダウンロード」をクリックすることにより、当社がお客様の個人情報をプライバシーポリシーに従い処理することを理解されたものとみなします。 プライバシーポリシー

以下の「ダウンロード」をクリックすることにより、お客様は以下に同意するものとします。 Confluent ライセンス契約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、お客様の個人データが以下に従い処理することにも同意するものとします: プライバシーポリシー

このウェブサイトでは、ユーザーエクスペリエンスの向上に加え、ウェブサイトのパフォーマンスとトラフィック分析のため、Cookie を使用しています。また、サイトの使用に関する情報をソーシャルメディア、広告、分析のパートナーと共有しています。