Project Metamorphosis: Unveiling the next-gen event streaming platformLearn More

Confluent CLI 1.0 is Now Generally Available for Cloud and Platform

Over a year ago, Confluent set out on a mission to improve user experience by empowering developers, operators, and architects with intuitive command line interfaces (CLIs) for managing their Confluent deployments, whether in the cloud or on prem. We engineered the ccloud and confluent CLIs from the ground up to be production-grade tools ready to run on a variety of platforms and to expose a rich set of features that make creating and maintaining an event streaming system as easy as a few commands. Having just reached the v1.0 milestone, with both CLIs declared generally available, we’re taking a few minutes to review some of the main features of the CLIs, as well as highlighting some of the brand new features you may not have explored yet.

Getting started with Confluent’s CLIs

Although the cloud and on-prem CLIs offer different feature sets, they’re built on the same codebase and have a very similar user experience (we’ve found this especially useful for hybrid customers who have to context switch between their on-prem and cloud deployments of the Confluent stack). To start, you can download both CLIs using a one-line terminal command or by downloading a tarball. For example, you can install the Confluent Cloud CLI via:

curl -L --http1.1 https://cnfl.io/ccloud-cli | sh -s -- -b /usr/local/bin

Both CLIs are available and officially supported on Windows, Mac, and Linux. Additionally, the Confluent Platform archives include a confluent CLI for that version of the platform. Hence, if you are an on-prem user, we recommend using the (potentially older) CLI versions bundled with the platform, since the latest bleeding-edge CLI versions may have incompatibilities. In the cloud, you can always update to the latest CLI version by running:

ccloud update

When updates are available, customers will be automatically notified in the CLI output once per day.

Once the CLI is installed, it’s easy to get started working with Apache Kafka®, Confluent Schema Registry, ksqlDB, Kafka Connect, and other components provided by Confluent. For instance, to create a new Kafka cluster in Confluent Cloud, check out the following demo (which also demonstrates machine-readable output):

Note that all commands in the CLIs follow the same command grammar:

<cli name> <resource> [<sub-resource>] <operation> [arguments and --flags]

And all commands have a --help flag to display usage information. For a great starter tutorial that uses the Cloud CLI to subsequently create API keys and produce to/consume from topics in a new cluster, see this tutorial by our rockstar Developer Relations Team. Note: if you’re already familiar with the basics, we recommend another tutorial that shows the power of the CLI for creating a complete ETL (extract, transform, load) pipeline in the cloud, including using the CLI to interact with Confluent Cloud ksqlDB and fully managed connectors.

Key Cloud features

As hinted above, the Confluent Cloud CLI unlocks a powerful set of features for both the day-to-day activities of a developer as well as managerial functions for operators, architects, and analysts. A sampling of features in the cloud includes:

  • Full CRUD operations on Kafka clusters and topics, including the management of clusters across all major cloud providers—Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP)
  •  Management of service accounts and their associated API keys and Kafka ACLs
  • Creation and control (including pausing or stopping) of connectors, including viewing all connector types that are available to create
  • Full CRUD operations for Schema Registry clusters, schemas, and subjects
  • Management and switching between environments
  • Administration of ksqlDB apps, including automatically configuring the appropriate ACLs

There are also features to make your interaction with the CLI easier (currently limited to bash or zsh shell users), such as autocompletion (ccloud completion --help) and a fully customizable PS1 prompt for adding ccloud information to your terminal (ccloud prompt --help).

Key on-prem features

The on-prem CLI experience is quite similar, though the feature set is tailored towards Confluent Platform deployments (though many such features, like Role-Based Access Control (RBAC), are planned for Confluent Cloud as well). Current features include:

  • Rich identity access management (IAM), currently in the form of RBAC and Kafka ACLs, and full management of centralized ACLs, RBAC roles, and RBAC role bindings (assignments of user and group principals to roles)
  • File-based secret management for Confluent Platform using envelope encryption
  • Cluster metadata discovery
  • “Local” commands for experimenting with a development instance of Confluent Platform running locally on your developer machine

New feature: Machine-readable output

A common request we received from users is that the table-formatted “human-readable” output generated by most CLI commands made it difficult to script around the CLIs, e.g., to automatically parse the output of CLI commands from shell or Python scripts in a production environment. In order to address the need for scriptability, commands in Confluent’s CLIs now accept the -o flag, with values such as json, yaml, or human. As an example, we can write this simple bash expression, which continuously ensures that our Kafka cluster hosted in Confluent Cloud is staying up (you can transparently verify we’re upholding our rock-solid 99.95% SLA!):

# Assumes cluster ID is lkc-abc123 (use `ccloud kafka cluster list` to get your cluster’s ID)
watch -n5 "ccloud kafka cluster describe lkc-abc123 -o json | jq '{status}[]' | xargs -I{} echo 'Your Kafka cluster is {}.'"

You can build on these simple ideas, such as querying the CLI to report information back to your company’s internal monitoring, alerting, and dashboard systems.

New feature: Long-lived authentication

Initially, the CLI copied the authentication model of Confluent’s GUI, which meant that all sessions timed out after an hour. However, this prevented the scriptability of the CLIs since it’s unreasonable to ask you to log in again every hour in your production environments. We designed a solution that handles each of the various types of authentication you can do with the CLIs:

  • A regular Confluent Cloud account (email and password)
  • An SSO-enabled Confluent Cloud account (authentication token returned from IdP)
  • An on-prem Confluent Server/metadata service instance (username and password)

In order to achieve consistent long-lived authentication for all of these mechanisms, we’ve added the --save flag to the login command of both ccloud and confluent. When used, the --save flag persists the relevant credentials to your local ~/.netrc file (~/_netrc on Windows). Then, if the CLI ever encounters an authentication error, it attempts to use the stored values to automatically re-authenticate and will silently succeed if matching credentials are found.

We note that, in addition to the above mechanisms, it’s also possible to log into a single Kafka cluster in Confluent Cloud using just a Kafka API key and secret (no Confluent Cloud credentials). The ccloud init command makes this possible and only offers the functionality of consuming from/producing to that cluster.

Closing thoughts and looking ahead

With both machine-readable output and long-lived authentication, Confluent’s CLIs are now able to be used in production environments for both interactive and automated use cases. Moreover, with support across a variety of operating systems and platform versions, the CLIs are a robust, reliable, developer-friendly way to interact with on-prem, cloud, or hybrid Confluent deployments. The CLIs are mature and stable enough to declare the v1.0 milestone; however, the CLIs are under active development, and the CLI Team has a number of exciting new features on the roadmap in the coming months. For example:

  • Integrating data from the forthcoming Confluent Cloud Metrics API into ccloud
  • Supporting RBAC management in the cloud, not just on prem, for fine-grained permissions management
  • Adding additional on-prem cluster discovery features to confluent

We’re excited for these features—and more we can’t announce yet! Keep an eye out for new features or updates released every week or two in the cloud, as well as with each new Confluent Platform release on prem.

Get in touch

The CLI Team is highly responsive to user feedback and is interested in hearing about both bug reports and feature requests—any way to help us serve our users better. If you would like to provide any suggestions, or if you’d just like to give some feedback, please feel free to file a ticket through Confluent Support, reach out on the #confluent-cli Slack channel, or send us an email at cli-team@confluent.io. We look forward to hearing from you!

David Hyde is the engineering lead for the CLI Team at Confluent.

Did you like this blog post? Share it now

Subscribe to the Confluent blog

More Articles Like This

Announcing ksqlDB 0.10.0

We’re excited to announce the release of ksqlDB 0.10.0, available now in the standalone distribution and on Confluent Cloud! This version includes a first-class Java client, improved Apache Kafka® key […]

Consistent Metastore Recovery for ksqlDB Using Apache Kafka Transactions

This is the second of a series of posts that dive deep into key improvements made to ksqlDB to prepare for production availability in Confluent Cloud. This post assumes familiarity […]

Kafka Streams Interactive Queries Go Prime Time

What is stopping you from using Kafka Streams as your data layer for building applications? After all, it comes with fast, embedded RocksDB storage, takes care of redundancy for you, […]

Sign Up Now

Start your 3-month trial. Get up to $200 off on each of your first 3 Confluent Cloud monthly bills

新規登録のみ。

上の「新規登録」をクリックすることにより、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします : プライバシーポリシー

上記の「新規登録」をクリックすることにより、お客様は以下に同意するものとします。 サービス利用規約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします: プライバシーポリシー

単一の Kafka Broker の場合には永遠に無料
i

商用版の機能を単一の Kafka Broker で無期限で使用できるソフトウェアです。2番目の Broker を追加すると、30日間の商用版試用期間が自動で開始します。この制限を単一の Broker へ戻すことでリセットすることはできません。

デプロイのタイプを選択
手動デプロイ
  • tar
  • zip
  • deb
  • rpm
  • docker
または
自動デプロイ
  • kubernetes
  • ansible

上の「無料ダウンロード」をクリックすることにより、当社がお客様の個人情報をプライバシーポリシーに従い処理することを理解されたものとみなします。 プライバシーポリシー

以下の「ダウンロード」をクリックすることにより、お客様は以下に同意するものとします。 Confluent ライセンス契約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、お客様の個人データが以下に従い処理することにも同意するものとします: プライバシーポリシー

このウェブサイトでは、ユーザーエクスペリエンスの向上に加え、ウェブサイトのパフォーマンスとトラフィック分析のため、Cookie を使用しています。また、サイトの使用に関する情報をソーシャルメディア、広告、分析のパートナーと共有しています。