Commit Graph

56 Commits (main)

Author SHA1 Message Date
Mike Kruskal ea81024ddf Refactor editions infrastructure to a top-level directory.
The only public target here is the edition defaults helper macro, which can be used by external runtimes and plugins.  None of this code is C++-specific though, and should be organized higher up.  Appropriate aliases are also placed at the top level for public targets

PiperOrigin-RevId: 625392504
2024-04-16 11:13:19 -07:00
Protobuf Team Bot 21d75f861c Remove references to rules_proto
They are not needed after the rules are move into protobuf repo.
Except for the reference to toolchain type, which is currently in rules_proto and can be moved after the implementation is moved into protobuf repo.

PiperOrigin-RevId: 622176865
2024-04-05 07:50:25 -07:00
Adam Cozzette c32c04c7ed Tweak BUILD setup to make the layering check change less disruptive
We recently updated the codebase to comply with the Bazel layering check, which
essentially requires any C++ header inclusion to be matched with a build
dependency on a target providing that header.

As part of that, I removed a handful of dependencies from the `//:protobuf`
target, since these dependencies were not set up in a way that respected the
layering check. However, I realized that this may cause a number of breakages,
especially since we did not provide the correct public targets until very
recently.

This change effectively adds back in the missing dependencies, so that projects
which do not yet adhere to the layering check can continue to depend on them
indirectly. This way, we still adhere to the layering check and make it
possible for projects that depend on us to do so, but in most cases we won't
immediately break anyone.

PiperOrigin-RevId: 607021111
2024-02-14 10:02:55 -08:00
aiuto 3ab1276bbb Update Bazel references to rules_pkg to the supported file locations. (#15780)
WIP:  I want to see this pass CI first.

Fixes: #15779

3 scripted transforms:
```
  -load("@rules_pkg//:mappings.bzl", ...)
  +load("@rules_pkg//pkg:mappings.bzl", ...)
```
```
  -load("@rules_pkg//:pkg.bzl", "pkg_tar")
  +load("@rules_pkg//pkg:tar.bzl", "pkg_tar")
```
```
  -load("@rules_pkg//:pkg.bzl", "pkg_zip")
  +load("@rules_pkg//pkg:zip.bzl", "pkg_zip")
```

Closes #15780

COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/15780 from aiuto:rules_pkg a630276341
PiperOrigin-RevId: 606641398
2024-02-13 09:08:42 -08:00
Adam Cozzette 220548f30c Add missing alias targets for C++ util libraries
Until recently, these targets were dependencies of `//:protobuf` and could thus
be accessed through that target. But now that we are adhering to the layering
check, we need to provide proper access in a way that respects that check. This
change uses top-level alias targets following the existing pattern.

PiperOrigin-RevId: 605123979
2024-02-07 15:57:49 -08:00
Sandy Zhang 65c65c2d04 Breaking Change: Use Editions features in Java full runtimes.
This change breaks compatibility with old generated code from previous major versions per the Cross Version Runtime policy: https://protobuf.dev/support/cross-version-runtime-guarantee. This includes old gencode from <4.26.x, which does not resolve features.

See https://protobuf.dev/news/2023-12-05/

PiperOrigin-RevId: 600487923
2024-01-22 09:57:26 -08:00
Adam Cozzette a7b0421c78 Breaking change: make protobuf comply with the C++ layering check
This check enforces that each C++ build target has the correct dependencies for
all headers that it includes. We have many targets that were not correct with
respect to this check, so I fixed them up.

I also cleaned up the C++ targets related to the well-known types. I created a
cc_proto_library() target for each one and removed the :wkt_cc_protos target,
since this was necessary to satisfy the layering check. I deleted the
//src/google/protobuf:protobuf_nowkt target and deprecated :protobuf_nowkt,
because the distinction between the :protobuf and :protobuf_nowkt targets was
not really correct. Neither one exposed the headers for the well-known types in
a way that was valid with respect to the layering check, and the idea of
bundling all the well-known types together is not idiomatic in Bazel anyway.
This is a breaking change, because the //:protobuf target no longer bundles the
well-known types. From now on they should be accessed through the new
//:*_cc_proto aliases in our top-level package.

I renamed the :port_def target to :port, which simplifies things a bit by
matching our internal name.

The original motivation for this change was that to move utf8_range onto our CI
infrastructure, we needed to make its dependency rules_fuzzing compatible with
Bazel 6. The rules_fuzzing project builds with the layering check, and I found
that the process of upgrading it to Bazel 6 made it take a dependency on
protobuf, which caused it to break due to layering violations. I was able to
work around this, but it would still be nice to comply with the layering check
so that we don't have to worry about this kind of thing in the future.

PiperOrigin-RevId: 595516736
2024-01-03 15:20:05 -08:00
Mike Kruskal 571b727cb6 Make rules_ruby a dev-only dependency.
There is no canonical rules_ruby repo today, and we don't want our fork to become one.  In order to unblock inclusion of Protobuf in the bzlmod registry, we're making this a dev dependency and dropping support for Bazel/Ruby.

Fixes #14569

PiperOrigin-RevId: 584393841
2023-11-21 12:10:30 -08:00
Mike Kruskal 898d8fa1fd Open-source editions Bazel rules for embedding defaults.
These utilities provide a way to embed a FeatureSetDefaults message into generators or runtimes that need to implement feature resolution.  They use protoc to handle the tricky reflection-based algorithm over feature protos, leaving only simple merges to be implemented in other languages.  See docs/design/editions/editions-life-of-a-featureset.md for more information.

PiperOrigin-RevId: 574554333
2023-10-18 12:23:27 -07:00
Adam Cozzette 501ececd39 Reorganize upb file structure
This change moves almost everything in the `upb/` directory up one level, so
that for example `upb/upb/generated_code_support.h` becomes just
`upb/generated_code_support.h`. The only exceptions I made to this were that I
left `upb/cmake` and `upb/BUILD` where they are, mostly because that avoids
conflict with other files and the current locations seem reasonable for now.

The `python/` directory is a little bit of a challenge because we had to merge
the existing directory there with `upb/python/`. I made `upb/python/BUILD` into
the BUILD file for the merged directory, and it effectively loads the contents
of the other BUILD file via `python/build_targets.bzl`, but I plan to clean
this up soon.

PiperOrigin-RevId: 568651768
2023-09-26 14:38:35 -07:00
Sandy Zhang 81068e8e8c Internal change
PiperOrigin-RevId: 566426899
2023-09-18 15:17:56 -07:00
Protobuf Team Bot 5b6c2459b5 Expose alias to protobuf_nowkt runtime
This will allow users such as Kythe to use the same runtime as the default toolchain in their own proto_lang_toolchain definitions.

PiperOrigin-RevId: 565438071
2023-09-14 12:08:26 -07:00
Adam Cozzette 7286ffceb2 Merge the protobuf and upb Bazel repos
A couple weeks ago we moved upb into the protobuf Git repo, and this change
continues the merger of the two repos by making them into a single Bazel repo.
This was mostly a matter of deleting upb's WORKSPACE file and fixing up a bunch
of references to reflect the new structure.

Most of the changes are pretty mechanical, but one thing that needed more
invasive changes was the Python script for generating CMakeLists.txt,
make_cmakelists.py. The WORKSPACE file it relied on no longer exists with this
change, so I updated it to hardcode the information it needed from that file.

PiperOrigin-RevId: 564810016
2023-09-12 13:09:54 -07:00
Mike Kruskal 4f9e41767a Release prototype of Protobuf Editions.
This represents the future direction of protobuf, replacing proto2/proto3 syntax with editions.  These will enable more incremental evolution of protobuf APIs through features, which are individual behaviors (such as whether field presence is explicit or implicit).  For more details see https://protobuf.dev/editions/overview/.

This PR contains a working implementation of editions for the protoc frontend and C++ code generation, along with various infrastructure improvements to support it.  It gives early access for anyone who wants to a preview of editions, but has no effect on proto2/proto3 syntax.  It is flag-guarded behind the `--experimental_editions` flag, and is an experimental feature with no guarantees.

PiperOrigin-RevId: 544805690
2023-06-30 20:18:49 -07:00
Mike Kruskal f51923041d Remove ODR violation from WKT codegen (#12406)
Closes #12406

COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/12406 from mkruskal-google:wkt 1c6748e6aa
PiperOrigin-RevId: 522418175
2023-04-06 13:19:50 -07:00
deannagarcia dbc918c6d0 Delete cmake/CMakeLists.txt in favor of CMakeLists.txt (#12295)
This fixes https://github.com/protocolbuffers/protobuf/issues/12263, but we will not cherrypick to 22.x since it could be breaking.

Closes #12295

COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/12295 from deannagarcia:deleteExtraCmakeFile 8fe901aee2
PiperOrigin-RevId: 518456432
2023-03-21 20:27:42 -07:00
Mike Kruskal 804ec94a16 Add release tests for statically linked binaries
For now, this only covers linux on the two architectures we have testing support for.  However, it serves as a good sanity check and can be expanded in the future.

PiperOrigin-RevId: 514449399
2023-03-06 10:26:21 -08:00
Mike Kruskal 5888882691 Internal change
PiperOrigin-RevId: 513329841
2023-03-01 13:46:08 -08:00
Mike Kruskal 723bd4c3c1 Modify release artifacts for protoc to statically link system libraries.
Closes #12063

PiperOrigin-RevId: 513034570
2023-02-28 14:13:48 -08:00
Xavier Bonaventura c8ebeb1733 Set incompatible_disallow_empty_glob in CI tests
In order to flip the flag, all downstream projects should be adapted. However, it is hard to fix them all if there are constant regressions. Adding it to the CI will ensure that once the project can build with incompatible_disallow_empty_glob it can keep building like that.
See: bazelbuild/bazel#15327
PiperOrigin-RevId: 507927389
2023-02-07 17:15:29 -08:00
Sandy Zhang 573be1d092 internal change
PiperOrigin-RevId: 506991260
2023-02-03 14:15:09 -08:00
Mike Kruskal 633e8f75d0 Bazelify ruby runtime (#10525)
This uses https://github.com/protocolbuffers/rules_ruby to fully Bazelify our ruby runtime code.  The Rakefile is left in place for now and is still used by our aarch64 tests.  With the current implementation ruby behaves similarly to our python wrapper, which selects whatever version is installed in the system.  Future enhancements will allow for more hermetic builds via Bazel flags to pin a specific version

Closes #10525

COPYBARA_INTEGRATE_REVIEW=https://github.com/protocolbuffers/protobuf/pull/10525 from mkruskal-google:rules_ruby 97fa1f70ab
PiperOrigin-RevId: 499283908
2023-01-03 12:12:47 -08:00
Protobuf Team Bot 64ae02c7fe Make labels relative
Prepare protobuf for bzlmod

Self-reference to `@com_google_protobuf` doesn't work with bzlmod.

In case this is a rule attribute, no reference is needed.

When the label is a default parameter value in a macro it needs to be wrapped with `Label` call, to relativise it correctly.

PiperOrigin-RevId: 496687786
2022-12-20 10:03:54 -08:00
Protobuf Team Bot 7787f3eafb Restore //:protoc_lib target.
This was removed in
cbd1adc6cf,
but it’s still present in the released versions of Protocol Buffers, and since
it’s a public target it’s part of the public API.

PiperOrigin-RevId: 495828746
2022-12-16 03:08:46 -08:00
Adam Cozzette f1998d91a8 Update auto-generation script to handle well-known types
I went ahead and deleted the update_file_list.sh script, because (a)
there was no good reason for it to be in a separate script and (b) we
now need to handle the well-known types in addition to file_lists.cmake.
With this change, we just invoke the staleness tests from the main
script to update everything.

While I was at it I made a couple small fixes:
- Don't skip the update step just because the previous commit was by
  "Protobuf Team Bot". Copybara commits use this name and we still want
  to do the auto-update step after them.
- Include the CL number in the description if the previous commit came
  from a CL.

PiperOrigin-RevId: 487231324
2022-11-09 06:57:08 -08:00
Mike Kruskal 32bea52ee6 Sync from Piper @mkruskal/protobuf-absl
PROTOBUF_SYNC_PIPER
2022-10-06 16:48:39 -07:00
Mike Kruskal 81e35132bf
Improve CMake installation tests (#10615)
* Using glob to remove headers instead of cyclic file_lists

* Simplify CMake config and include missing files

* Don't remove generated proto headers

* Fix broken CMake proto dependencies instead of opting out pb.h files.

* Fixing cyclic dependency
2022-09-19 11:08:21 -07:00
Mike Kruskal 0264866ce6
Use generated WKT code in Bazel builds (#10576)
* Use generated WKT code in Bazel builds

* Prefer src over external for genrule

* Prefer external over src for genrule

* Proper fix for windows proto path issues
2022-09-14 19:33:54 -07:00
Mike Kruskal d220b43c6c
Enable warnings as errors during tests (#10482)
* Enable warnings as errors by default for test builds

* Fixing C++ warnings

* Adding host flags, and enabling warnings as error for non-C++ too

* Switch to BUILD copts instead of bazelrc to treat Windows as a snowflake

* Disable warnings as errors on Windows, since it doesn't like the c++14 flag
2022-08-29 18:19:38 -04:00
Mike Kruskal ba6d17354f Fixing reordered proto library includes 2022-08-12 10:44:59 -07:00
Mike Kruskal ca4b0630a6 Sync from Piper @467099277
PROTOBUF_SYNC_PIPER
2022-08-11 20:55:01 -07:00
Mike Kruskal ed5c57a574
Remove all autotools usage (#10132)
* Bazelfying conformance tests

Adding infrastructure to "Bazelify" languages other than Java and C++

* Delete benchmarks for languages supported by other repositories

* Bazelfying benchmark tests

* Bazelfying python

Use upb's system python rule instead of branching tensorflow

* Bazelfying Ruby

* Bazelfying C#

* Bazelfying Objective-c

* Bazelfying Kokoro mac builds

* Bazelfying Kokoro linux builds

* Deleting all deprecated files from autotools cleanup

This boils down to Makefile.am and tests.sh and all of their remaining references

* Cleanup after PR reorganizing

- Enable 32 bit tests
- Move conformance tests back
- Use select statements to select alternate runtimes
- Add internal prefixes to proto library macros

* Updating READMEs to use bazel instead of autotools.

* Bazelfying Kokoro release builds

* First round of review fixes

* Second round of review fixes

* Third round of review fixes

* Filtering out conformance tests from Bazel on Windows (b/241484899)

* Add version metadata that was previously scraped from configure.ac

* fixing typo from previous fix

* Adding ruby version tests

* Bumping pinned upb version, and adding tests to python CI
2022-08-10 22:51:29 -07:00
Adam Cozzette 41762566dd Merge branch '21.x' into merge-21-x 2022-07-21 23:38:04 +00:00
Yash Tibrewal c8b4ef27a0
[Bazel] Add back a filegroup for :well_known_protos (#10279) 2022-07-20 09:27:31 -07:00
Mike Kruskal 445a49480c
Adding Github Action to keep CMake generated file lists up to date (#10243) 2022-07-11 22:04:40 -07:00
YJ 2bc5f95fe3
add missing quotes (#10123) 2022-06-09 13:27:39 -07:00
Philipp Stephani 226dd65989 Make //:protobuf_python and //:well_known_types_py_pb2 public.
These targets form the public interface of the Python protocol buffer support
and must always be public.  It looks like commit
a6901f057e accidentally restricted their
visibility.
2022-06-08 22:18:57 +02:00
bttk 85fcd02e32
Remove merge markers from BUILD.bazel and fix incorrect merge (#10084)
* Remove merge markers from BUILD.bazel

* fix build errors
2022-05-31 15:34:59 -04:00
Sandy Zhang 15e09f56b0 Fix unsaved merge resolution 2022-05-28 00:17:14 +00:00
Sandy Zhang 258514019f Merge 21.x into main 2022-05-28 00:06:34 +00:00
David L. Jones ceaae1b6a3
[Bazel] Add back a filegroup for :well_known_protos. (#10061)
This was removed in #9915, since it is misleadingly named (the set of sources is "well known," but not the same as the Well-Known Types).
2022-05-26 16:32:15 -07:00
Joshua Haberman 2dc7679cc1 Fixed the Windows build of protoc by statically linking deps. 2022-05-23 20:59:31 +00:00
David L. Jones 171a6b1155
[Bazel/C++] Move core C++ implementation to //src/google/protobuf package. (#9988)
This is somewhat of a rough cut, since it doesn't split apart the lite and full targets, or unit tests. However, it does split sources under src/google/protobuf into a separate package, which is a fairly impactful change.
2022-05-18 13:45:22 -07:00
David L. Jones 07303d6f54
[Bazel/C++] Factor out util library (#9987)
This change creates new Bazel packages for the C++ util libraries (similar to #9980 and #9985).
2022-05-17 18:13:22 -07:00
David L. Jones cbd1adc6cf
[Bazel/C++] Factor out compiler packages. (#9985)
This change adds packages for protoc sources under src/google/protobuf/compiler (this is similar to what was done in #9980).
2022-05-17 16:39:47 -07:00
David L. Jones c2c770e7ea
[Bazel/C++] Factor out stubs, io, and testing libraries. (#9980)
This change creates packages under src/google/protobuf/{io,stubs,testing} and moves build definitions there. Future changes will handle .../util and .../compiler, and finally src/google/protobuf.
2022-05-17 13:26:46 -07:00
Deanna Garcia 0f3ee3b625 Fix protoc_release target 2022-05-16 22:48:41 +00:00
David L. Jones b3cbea18ea
[Bazel] Move Python rules to //python (#9952) 2022-05-12 19:48:58 -07:00
David L. Jones b64cd5ac64
Switch C++ test protos to use the normal cc_proto_library rules. (#9950) 2022-05-11 16:45:17 -07:00
David L. Jones 354aba886e
Add a test for bazel-built dist archives (#9929)
This also fixes some packaging rules that are needed for the result to be buildable.
2022-05-09 16:32:59 -07:00