I don't know whether github actions output can be tampered with by you, but the only actually reliable way (that I know of) to prove that your binaries correspond to a certain state of the sourcecode is to support reproducible builds (See e.g. https://reproducible-builds.org/).
All other methods require trust (in either the developer or w.r.t. github actions towards github).
The drawback is of course, that to verify whether your binaries are good, someone needs to rebuild the software, but it is a good tool to build and maintain trust in your signed binaries, especially if they deal with sensitive information like private keys.