Add support for protobuf version 4.29.x #3175

Activity

1. ilanKeshet

   added

   Type: enhancement

   on Mar 13, 2025
2. wgtmac commented on Mar 14, 2025

   

   wgtmac

   on Mar 14, 2025

   Member

   More actions

   @Fokko Is it safe to bump the major version of protobuf?
3. ilanKeshet commented on Mar 23, 2025

   

   ilanKeshet

   on Mar 23, 2025

   Author

   More actions

   According to discussion on #1308 (comment) the parquet-java project would have to be upgraded to Java-11 before we could support protobuf version 4.x
4. code4dc commented on May 6, 2025

   

   code4dc

   on May 6, 2025

   More actions

   Is there a ticket out there to update to Java 11?
5. uwemaurer commented on Oct 14, 2025

   

   uwemaurer

   on Oct 14, 2025

   Last edited by uwemaurer

   Contributor

   More actions

   We were able to upgrade to protobuf 4.32.1 and only had to replace ProtoWriteSupport.java with a patched version, using the patch below:

   @@ -445,10 +445,14 @@
    
        private void writeAllFields(MessageOrBuilder pb) {
          Descriptor messageDescriptor = pb.getDescriptorForType();
   -      Descriptors.FileDescriptor.Syntax syntax =
   -          messageDescriptor.getFile().getSyntax();
   +      String syntax =
   +          messageDescriptor.getFile().toProto().getSyntax();
   +      if ("editions".equals(syntax)) {
   +          throw new UnsupportedOperationException("protocol buffers 'editions' not supported");
   +      }
   +      boolean isProto2 = !"proto3".equals(syntax);
    
   -      if (Descriptors.FileDescriptor.Syntax.PROTO2.equals(syntax)) {
   +      if (isProto2) {
            // Returns changed fields with values. Map is ordered by id.
            Map<FieldDescriptor, Object> changedPbFields = pb.getAllFields();
    
   @@ -464,7 +468,7 @@
              int fieldIndex = fieldDescriptor.getIndex();
              fieldWriters[fieldIndex].writeField(entry.getValue());
            }
   -      } else if (Descriptors.FileDescriptor.Syntax.PROTO3.equals(syntax)) {
   +      } else {
            List<FieldDescriptor> fieldDescriptors = messageDescriptor.getFields();
            for (FieldDescriptor fieldDescriptor : fieldDescriptors) {
              FieldDescriptor.Type type = fieldDescriptor.getType();
   @@ -797,4 +801,3 @@
        throw new InvalidRecordException(exceptionMsg);
      }
    }
   

   This code should actually also work with the protobuf 3 library. The problem was that in protobuf 4 the getFile().getSyntax() method is not there anymore, but getFile().toProto().getSyntax() works (returns a String instead of enum)
6. code4dc commented on Oct 19, 2025

   

   code4dc

   on Oct 19, 2025

   More actions

   @uwemaurer Can you create a PR for this? Maybe they'll agree to merge your patch for the next release.

   👍React with 👍1Reacted by Uwe Maurer

   🚀React with 🚀1Reacted by Fokko Driesprong
7. Fokko commented on Oct 22, 2025

   

   Fokko

   on Oct 22, 2025

   Contributor

   More actions

   @uwemaurer That's great! Can you create a PR so you'll be credited for the work? We can easily test to make sure that it still works against Protobuf 3 by first doing the patch and then bumping protobuf in a separate PR.

   👍React with 👍1Reacted by Uwe Maurer
8. uwemaurer commented on Oct 23, 2025

   

   uwemaurer

   on Oct 23, 2025

   Contributor

   More actions

   yes I will prepare a PR
9. uwemaurer

   added a commit that references this issue on Oct 23, 2025

   support protobuf library version 4

   ...

   a814e33
10. Fokko

    added a commit that references this issue on Nov 19, 2025

    GH-3175: support protobuf library version 4 (#3352)

    ...

    a6bad6a
11. Fokko

    closed this as completedin #3352on Nov 19, 2025