Check your deed to see how the easement is described. They’re almost always limited in scope (for example, automobile ingress/egress to a neighbor’s landlocked property) and it’s highly unlikely that that scope includes political signage. Once you verify, pull the sign out and give it back to your neighbor. It is very common for easements to get confused with public land, that’s almost certainly what happened here.