It wasn't like he wanted Thor to be banished. If you didn't notice, he tried to stand up for Thor, but Odin just told him to shut up. Even though he was angry at Thor a lot, he never hated him, even when he convinced himself that he did.
This man is awesome! Tom Hiddleston continues to treat ladies with respect while others are so negative about them. I hope a lot of people look up to him and realize that this is how to treat someone to get respect