Tell you what, I suggest having a nice read through this comprehensive piece: https://plato.stanford.edu/entries/information-entropy/ You will find parts that tend to your pov - and other parts that don't.