Skip to content

Support protobuf native schema#486

Open
shibd wants to merge 4 commits into
apache:masterfrom
shibd:protobuf_native
Open

Support protobuf native schema#486
shibd wants to merge 4 commits into
apache:masterfrom
shibd:protobuf_native

Conversation

@shibd
Copy link
Copy Markdown
Member

@shibd shibd commented May 25, 2026

Fixes #

Master Issue: #

Motivation

Support protobuf native schema

Modifications

  1. Support protobuf native schema
  2. Provide tools to help the user create schema info for the protobuf native schema.

Verifying this change

  • Added protobuf schema end-to-end test

Documentation

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

@shibd shibd self-assigned this May 25, 2026
@shibd shibd added this to the 1.18.0 milestone May 25, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class support for Pulsar Protobuf Native schema in the Node client by exposing helper utilities to generate the required schema info from protobufjs descriptor output, updating the native binding schema plumbing, and adding an end-to-end test validating produce/consume behavior.

Changes:

  • Add ProtobufNativeSchema helper module (root creation from reflection JSON + schema info generation using FileDescriptorSet).
  • Extend schema type handling in the native binding to support ProtobufNative.
  • Add an end-to-end Jest test (with generated protobuf artifacts) and include protobufjs as a dependency.

Reviewed changes

Copilot reviewed 7 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/SchemaInfo.h Switch schema representation to Pulsar C++ SchemaType + std::map properties.
src/SchemaInfo.cc Add ProtobufNative mapping and set schema on producer/consumer configs via C++ API.
src/ProtobufNativeSchema.js New helper utilities to build Protobuf Native schema info from protobufjs.
index.js Export ProtobufNativeSchema from the main module.
index.d.ts Add ProtobufNativeSchema typings and extend SchemaType union.
package.json Add protobufjs dependency.
yarn.lock / package-lock.json Lockfile updates for protobufjs (and transitive deps).
.eslintignore Ignore generated protobuf artifacts under tests.
tests/protobuf_native_schema.test.js New end-to-end test for Protobuf Native schema produce/consume.
tests/protobuf_schema/generated/* Generated protobuf JS/TS artifacts + reflection JSON used by the test.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/SchemaInfo.cc
Comment thread src/SchemaInfo.cc
Comment thread src/ProtobufNativeSchema.js Outdated
Copy link
Copy Markdown
Contributor

@BewareMyPower BewareMyPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you verify if it can work with Java client? You can fork https://github.com/BewareMyPower/pulsar-protobuf-native-demo to test the latest implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants