Edit: obligatory explanation (thanks mods for squaring me away)…
What you see via the UI isn’t “all that exists”. Unlike Reddit, where everything is a black box, there are a lot more eyeballs who can see “under the hood”. Any instance admin, proper or rogue, gets a ton of information that users won’t normally see. The attached example demonstrates that while users will only see upvote/downvote tallies, admins can see who actually performed those actions.
Edit: To clarify, not just YOUR instance admin gets this info. This is ANY instance admin across the Fediverse.
Tbh it would be trivial to just salt and hash the usernames (for keying the votes), no need to encrypt or involve the users password. The salting and hashing would be handled by the users home instance ( which presumably the user trusts ) so building a rainbow table would be non trivial for an attacker ( assuming the home instance keeps its salts secret ).
I like this idea. Easily solves the main issue with other instance admins getting access to it, while also being easy to implement.
Another option would be to aggregate votes per instance so programming.dev might only see “42 upvotes from lemmy.world”, but not user details. I don’t think that changes the inter-instance trust equation, at least not notably, and it even works in conjunction with non-aggregated upvotes and displaying everything publicly.