Commit Graph

1063 Commits (main)

Author SHA1 Message Date
Sandy Zhang 11c27dfa4b Lazily resolve features for proto2 and proto3 for compatibility with old open source gencode that does not invoke feature resolution from gencode static init.
PiperOrigin-RevId: 634804242
2024-05-17 09:44:20 -07:00
Protobuf Team Bot a521e6e152 Internal chanegs
PiperOrigin-RevId: 634600978
2024-05-16 19:12:25 -07:00
Protobuf Team Bot 24751b405a Add legacy Java TextFormat APIs
PiperOrigin-RevId: 634047851
2024-05-15 13:06:14 -07:00
Protobuf Team Bot 05781615a1 Internal changes/optimizations
PiperOrigin-RevId: 633982805
2024-05-15 09:39:15 -07:00
Protobuf Team Bot dde03553c9 Internal change
PiperOrigin-RevId: 633679764
2024-05-14 13:03:23 -07:00
Protobuf Team Bot bb68eb22dd Implement new Debug API with redaction.
Implement emittingSingleLine TextFormat printer option.

PiperOrigin-RevId: 633672722
2024-05-14 12:40:20 -07:00
Mark Hansen c6e2778f91 Replace SmallSortedMap.EmptySet with equivalent Collections.emptySet()
This reduces our code weight by a little (3 classes).

Collections.emptySet also has a singleton empty iterator, so it doesn't allocate.

PiperOrigin-RevId: 633667264
2024-05-14 12:19:26 -07:00
Mark Hansen f7a866a9ea Make IntArrayList consistent with other primitive array lists, using code generator
PiperOrigin-RevId: 633042614
2024-05-12 17:10:45 -07:00
Sandy Zhang c99cf4bbfa Add `java` to reserved names to escape extensions to `java_`. This may break existing references to gencode for extensions named `java`.
This prevents shadowing of `java.lang` package commonly used in protobuf gencode. Existing extensions named `java` may or may not previously fail to compile depending on if the contents of their .proto result in gencode using `java.lang`. This is needed to fix `java_features.proto` lite gencode since enum gencode uses `java.lang`. Fields named `java` should already be escaped.

*Warning: This may break user code for existing protos with extensions named `java`. References to the extension should be renamed to use `java_` e.g. registry.add(GeneratedClassName.java_)*

PiperOrigin-RevId: 632508249
2024-05-10 08:49:22 -07:00
Mark Hansen 4b7c2d4381 Avoid allocating TreeMap in UnknownFieldSet.Builder.asMap
If UnknownFieldSet is empty

PiperOrigin-RevId: 632504647
2024-05-10 08:42:39 -07:00
Mark Hansen 8c6e5b7963 Avoid allocating iterator in TextFormat printing UnknownFieldSet
PiperOrigin-RevId: 632503842
2024-05-10 08:38:55 -07:00
Protobuf Team Bot fefec68753 Internal changes/optimizations
PiperOrigin-RevId: 632294209
2024-05-09 15:54:16 -07:00
Protobuf Team Bot af1ecc0353 Add a check to ensure that the byte limit is not negative.
PiperOrigin-RevId: 632284538
2024-05-09 15:18:31 -07:00
Protobuf Team Bot 5584c934bf Update AbstractMessageLite.addAll so that it won't throw when input is a LazyStringArrayList with byte[] elements.
Accidentally run into an error when I'm doing protoA.addAllFieldA(protoB.getMutableFieldAList())

PiperOrigin-RevId: 631257710
2024-05-06 19:11:03 -07:00
Mark Hansen c56bddd707 Add UnknownFieldSet.isEmpty
Allows callers to check if field set is empty without cloning the map

PiperOrigin-RevId: 631237925
2024-05-06 17:38:35 -07:00
Mark Hansen d0b016fcfc Java full runtime: Avoid allocating when calling asMap on empty UnknownFieldSet
PiperOrigin-RevId: 631235442
2024-05-06 17:28:34 -07:00
Protobuf Team Bot f597991857 Return the default instance when parsing from an empty byte[].
PiperOrigin-RevId: 631213831
2024-05-06 16:05:29 -07:00
Sandy Zhang 4c8da99c48 Deprecate building from source with Maven to be removed in 4.28.x.
Most users should use the *pre-built* artifacts, or should build from source with Bazel instead.

PiperOrigin-RevId: 631187695
2024-05-06 14:36:12 -07:00
Mark Hansen 4f12891b87 Java full runtime: Avoid allocating iterator if UnknownFieldSet's TreeMap is empty
I also considered using Collections.emptyNavigableMap(), which I thought might
use a specialized allocation free empty iterator, but it allocates, and its
clone() method isn't nicely exposed.

PiperOrigin-RevId: 630234849
2024-05-02 18:15:32 -07:00
Éamonn McManus 39a1c6a46e In the Kotlin DSL, reference builder methods with property syntax.
Instead of _builder.getFoo() and _builder.setFoo(value), the generated code now uses _builder.foo and _builder.foo = value. When compiling against the Java Proto API, this makes no difference, since the Kotlin compiler treats Java methods getFoo/setFoo as defining a Kotlin property foo. But if a compatible proto API is implemented in Kotlin then there is no such equivalence. Such an implementation would have to define either both forms or just the one that the DSL uses. For a Kotlin API it is more natural to use properties.

Similarly, the generated Java methods getFoosList(), getFoosCount(), getFoosMap(), and getEnumValue() are accessed via property names (foosList, foosCount, foosMap, enumValue).

(This is the second application of this change. The first had to be rolled back because it didn't handle conflict-resolution names correctly.)

PiperOrigin-RevId: 629807104
2024-05-01 12:49:52 -07:00
Mark Hansen fb054c8e41 Java Proto lite: avoid boxing Integers accessing enum lists
Provide a specialization for ListAdapter that avoids boxing ints into Integers.

Use this for repeated enum accessors.

In practice, the ints for most enums will fit into the JDK's "boxed int cache", avoiding extra allocations. For large enum numbers, they won't fit, and would allocate. Removing the boxing should also slightly speed up code.

PiperOrigin-RevId: 629583247
2024-04-30 18:09:30 -07:00
Éamonn McManus c24e7529eb Automated rollback of commit e9c9970eb4.
PiperOrigin-RevId: 629405086
2024-04-30 07:32:38 -07:00
Éamonn McManus e9c9970eb4 In the Kotlin DSL, reference builder methods with property syntax.
Instead of `_builder.getFoo()` and `_builder.setFoo(value)`, the generated code now uses `_builder.foo` and `_builder.foo = value`. When compiling against the Java Proto API, this makes no difference, since the Kotlin compiler treats Java methods `getFoo`/`setFoo` as defining a Kotlin property `foo`. But if a compatible proto API is implemented in Kotlin then there is no such equivalence. Such an implementation would have to define either both forms or just the one that the DSL uses. For a Kotlin API it is more natural to use properties.

Similarly, the generated Java methods `getFoosList()`, `getFoosCount()`, `getFoosMap()`, and `getEnumValue()` are accessed via property names (`foosList`, `foosCount`, `foosMap`, `enumValue`).

PiperOrigin-RevId: 629220383
2024-04-29 16:30:52 -07:00
Protobuf Team Bot de6aba9f63 Java Proto Lite: Serialize repeated enums without allocating
Serialize repeated enums without allocating

PiperOrigin-RevId: 628915124
2024-04-28 17:30:44 -07:00
Protobuf Team Bot eaee04d897 Avoid boxing when serializing primitive lists in CodedOutputStreamWriter
Previously, these List.get methods returned Object, forcing boxing, and (if not in the primitive's box cache) allocation, with all the cost that entails.

I've taken the approach of duplicating methods to specialise for primitives, like BinaryWriter does for non-lite protos.

I initially considered checking the class of the list on every iteration of the
loop, but that feels wasteful, when we can
check it once at the start of the loop. This also means we have the same
behaviour as serverside protos. At the cost of a few more methods, but
hopefully they're trivially inlineable, so hopefully leading to a small dex increase without really increasing the method count.

Given this is a public API, I don't think we can remove the List<Long> overloads either.

PiperOrigin-RevId: 627183583
2024-04-22 16:12:47 -07:00
Protobuf Team Bot 971f88aa63 Improve C++, Python, Java runtime version representations to avoid merge conflicts during release.
PiperOrigin-RevId: 626480269
2024-04-19 15:21:27 -07:00
Mike Kruskal 95877d8569 Updating version.json and repo version numbers to: 28-dev (#16555)
Closes #16555

PiperOrigin-RevId: 626081638
2024-04-18 10:47:14 -07:00
Mike Kruskal 57a6e8df1c Add dedicated tests to lock down that overridable defaults never change.
Once an edition is released these should be fixed forever.  On the other hand, the fixed defaults may see new features included as we change legacy behaviors in future editions.

PiperOrigin-RevId: 625842071
2024-04-17 16:36:06 -07:00
Protobuf Team Bot 6afdd5a135 Auto-generate files after cl/625815212 2024-04-17 22:09:39 +00:00
Protobuf Team Bot c755d9ac1a Add `load()` statements for the Bazel builtin top-level java symbols
Loads are being added in preparation for moving the symbols out of Bazel and into `rules_java`.

PiperOrigin-RevId: 625688195
2024-04-17 08:18:16 -07:00
Mike Kruskal db5814e4ff Use newly split feature defaults in plugins and runtimes.
The new fields fixed_features and overridable_features can be simply merged to recover the old aggregate defaults.  By splitting them though, plugins and runtimes get some extra information about lifetimes for enforcement.

PiperOrigin-RevId: 625527117
2024-04-16 19:18:40 -07:00
Mike Kruskal ea81024ddf Refactor editions infrastructure to a top-level directory.
The only public target here is the edition defaults helper macro, which can be used by external runtimes and plugins.  None of this code is C++-specific though, and should be organized higher up.  Appropriate aliases are also placed at the top level for public targets

PiperOrigin-RevId: 625392504
2024-04-16 11:13:19 -07:00
Protobuf Team Bot a5be8bdb9f Auto-generate files after cl/625084569 2024-04-15 21:45:12 +00:00
Mike Kruskal 7d87a1780a Hook up staleness tests to non-bootstrap edition defaults
PiperOrigin-RevId: 623936729
2024-04-11 14:23:41 -07:00
Mike Kruskal 77f62de93e Define lifetimes of edition 2023 features
PiperOrigin-RevId: 623690798
2024-04-10 20:33:20 -07:00
Protobuf Team Bot a94f57bd69 Check that size is non-negative when reading string or bytes in StreamDecoder.
This ensures that StreamDecoder throws a InvalidProtocolBufferException instead of an IllegalStateException on some invalid input.

All other implementations of CodedInputStream already do this check.

PiperOrigin-RevId: 623383287
2024-04-09 23:00:36 -07:00
Mike Kruskal 29c69ff00b Fix text-format delimited field handling
This updates all our text parsers and serializers to better handle tag-delimited fields under editions.  Under proto2, groups were the only tag-delimited fields possible, and the group name (i.e. the message type) was guaranteed to be unique.  Text-format and various generators used this instead of the synthetic field name (lower-cased group name) to represent these fields.

Under editions, we've removed group syntax and allowed any message field to be tag-delimited.  This breaks those cases when adding new tag-delimited fields where the message type might not be unique or correspond to the field name.  Code generators have already been fixed to treat "group-like" fields using the old behavior, and treat new fields like any other sub-message.

This change addresses the text-format issue.  Text parsers will accept *either* the type or field name for "group-like" fields, and only the field name for every other message field.  Text serializers will continue to emit the message name for "group-like" fields, but also use the field name for everything else.

This creates some awkward capitalization behavior for fields that happen to *look* like proto2 groups, but it won't lead to any conflicts or invalid encodings.  A feature will likely be added to edition 2024 to allow for migration off this legacy behavior.

PiperOrigin-RevId: 622260327
2024-04-05 13:08:43 -07:00
Protobuf Team Bot 21d75f861c Remove references to rules_proto
They are not needed after the rules are move into protobuf repo.
Except for the reference to toolchain type, which is currently in rules_proto and can be moved after the implementation is moved into protobuf repo.

PiperOrigin-RevId: 622176865
2024-04-05 07:50:25 -07:00
Protobuf Team Bot 1ebec9a4c3 Add `load()` statements for the builtin Bazel java rules
Loads are being added in preparation for moving the rules out of Bazel and into `rules_java`.

PiperOrigin-RevId: 621541265
2024-04-03 09:10:11 -07:00
Protobuf Team Bot 83ec410d30 Internal change
PiperOrigin-RevId: 619377672
2024-03-27 05:50:55 -07:00
Sandy Zhang 6c45efd297 Fix validateFeatures to be called after resolved features are actually set to this.features
PiperOrigin-RevId: 619296935
2024-03-26 13:42:52 -07:00
Sandy Zhang 274bc7f26f Move java_features.proto to java/core/srce/main/resources
This also fixes maven to package this correctly as google/protobuf/java_features.proto (same dir as WKT/descriptor.proto) instead of com/google/protobuf/java_features.proto.

Fixes https://github.com/protocolbuffers/protobuf/issues/16155

PiperOrigin-RevId: 619015714
2024-03-25 18:01:21 -07:00
Liam Miller-Cushon b36458fd80 Make `NioByteString` private member class of `ByteString`
PiperOrigin-RevId: 618910772
2024-03-25 11:44:46 -07:00
Sandy Zhang 9ce51d1ed8 Set label to REQUIRED for descriptors with LEGACY_REQUIRED feature.
Ensures isOptional() does not return true for LEGACY_REQUIRED fields which would otherwise get the optional label applied by default (non-optional fields still get the optional label).

Adds validation to feature resolution instead of cross link, which is too early to have FieldPresence.LEGACY_REQUIRED resolved.

PiperOrigin-RevId: 618857590
2024-03-25 08:59:59 -07:00
Sandy Zhang 369732a581 Add java_features.proto to kotlin_mvn release
PiperOrigin-RevId: 618295219
2024-03-22 15:11:06 -07:00
Sandy Zhang 500c953d10 Enable editions support for Java Lite.
java_features.proto and descriptor.proto should now both be included in the Java Lite and Kotlin Lite maven releases.

Fixes https://github.com/protocolbuffers/protobuf/issues/7331

PiperOrigin-RevId: 618251348
2024-03-22 12:13:06 -07:00
Protobuf Team Bot 821e854b5a Automated Code Change
PiperOrigin-RevId: 617984990
2024-03-21 15:45:34 -07:00
Mike Kruskal 0e167e796f Disallow all feature types except enum and boolean.
Features are designed as temporary migration tools, and any unbounded type leaves the system open to unexpected use.  Features should have a fixed set of values, with well defined behaviors.

PiperOrigin-RevId: 617933544
2024-03-21 12:51:15 -07:00
Sandy Zhang 925de189ec Fix mistake in osgi.bzl to actually use list source_jars.
Bazel 7 updated java_out.source_jars to return a Depset.

Release images are still on Bazel 6, but cloudtops were updated to Bazel 7 already.

PiperOrigin-RevId: 617253874
2024-03-19 12:14:37 -07:00
Liam Miller-Cushon 2c07ea5317 Suppress a warning about a class initializer cycle
PiperOrigin-RevId: 616953542
2024-03-18 14:49:45 -07:00