Skip to content

Streamlit UI

A web interface for selecting schemas, editing them, previewing generated data, and producing directly to Kafka.

Quick start

pip install "avro-datagen[ui]"
avro-datagen ui

Opens at http://localhost:8501. No clone needed -- the app, example schema, and theme are bundled in the pip package.

Options

avro-datagen ui                              # default port 8501
avro-datagen ui --port 3000                  # custom port
avro-datagen ui --schema-dir ./my-schemas    # use your own schemas
avro-datagen ui --kafka                      # enable Kafka producer section
Flag Default Description
--port 8501 Port to run the UI on
--schema-dir bundled schemas Directory containing .avsc files
--kafka off Enable Kafka sidebar and producer section

With Kafka

By default the UI has no Kafka dependency. To enable the Kafka connection sidebar and produce section:

pip install "avro-datagen[app]"    # includes confluent-kafka
avro-datagen ui --kafka

Schema resolution

The UI looks for schemas in this order:

  1. --schema-dir flag (if provided)
  2. SCHEMA_DIR environment variable (if set)
  3. ./schemas/ in the current directory (if it exists)
  4. Bundled example schemas from the package

Read-only schema directories

Mounting the schema directory read-only is a supported setup — it is the right one when schemas are git-owned contracts that should not be edited from a web form:

docker run -p 8501:8501 -v "$PWD/schemas:/schemas:ro" -e SCHEMA_DIR=/schemas ...

The UI detects this at startup, disables the save controls with an explanatory hint, and shows a banner. Browsing, sampling, generating and the editor's Download button all keep working — only writing back is off.

Features

JSON format

A Wire / Human toggle above the schema tabs controls how temporal logical types are displayed:

  • Wire (default) — the Avro physical encoding, e.g. 1785691423392
  • Human — ISO-8601, e.g. "2026-08-02T17:23:43.392Z"

Human form is what a Kafka UI produce form with a schema-registry serde expects, so a sample copied in that mode can be pasted straight in. The toggle affects the sample record, the generated records table and the download only — the Kafka producer always sends wire form. The CLI equivalent is --json-format human.

Schema tabs

Tab Description
Local schemas Pick from .avsc files. Collapsible JSON view + sample record
Upload Drop any .avsc or .json file. Collapsible JSON view + sample record
Editor Full JSON editor. Live sample record updates as you type. Save to file or download

Generate

  • Set record count and seed
  • Records generated in one batch, displayed as a table
  • Metrics: count, generation rate (rec/s), elapsed time
  • Expandable "How fields are resolved" section explains the resolution path for each field

Kafka Produce

  • Configure connection, auth, and tuning via the sidebar
  • Real-time progress with Produce / Stop controls
  • Live metrics: sent, errors, elapsed

Environment variables

All sidebar fields can be pre-filled via environment variables or a .env file:

Field Environment variable Default
Bootstrap servers KAFKA_BOOTSTRAP_SERVERS localhost:29092
Topic KAFKA_TOPIC transactions
Security protocol KAFKA_SECURITY_PROTOCOL (none)
SASL mechanism KAFKA_SASL_MECHANISM (none)
Username KAFKA_SASL_USERNAME (none)
Password KAFKA_SASL_PASSWORD (none)
Linger (ms) KAFKA_LINGER_MS 5
Batch size KAFKA_BATCH_SIZE 16384
Compression KAFKA_COMPRESSION_TYPE none
Schema directory SCHEMA_DIR (bundled)

Valid values for KAFKA_COMPRESSION_TYPE: none, gzip, snappy, lz4, zstd. An unknown value falls back to none.

Schema editor

The editor tab lets you modify the full schema JSON. When the JSON is valid:

  • A sample record is rendered below
  • Save to file writes back to the original .avsc (or schemas/ for new schemas)
  • Download exports as a .avsc file
  • Reset reverts to the originally loaded schema

The edited schema is used by both Generate and Kafka Produce sections.