Gustavo Lacerda

Some physicists used to toy with the notion that entropy is subjective and only refers to our "lack of knowledge" about a system. But from what I understand, computer simulations put and end to that way of thinking. Once it became possible to simulate a large number of interacting objects bouncing around exchanging energy, it became obvious that the entropy of a system is an objective quantity which refers to how many accessible states there are in the system.

I briefly looked over the thread, and I noticed you asked "is entropy defined for a macroscopic particle". I think I know what you're asking here, but I'm not sure so correct me if I misinterpretted. I think what you're asking is: does entropy apply to classical systems which have no constituants small enough for quantum mechanics to play a role? And the answer is yes. Entropy is an entirely classical concept; the theory of it was mostly sorted out before quantum mechanics was discovered. In the end many questions about what's going to happen to the entropy of the universe (if such a quantity really makes sense to talk about) may be intimately tied to quantum mechanics, but that's a somewhat separate subject.

Another thing to realize about entropy, is that you need a very large number of states in order to define it. And you also need the system to be in equilibrium. If there are only a small number of states (like... less than a million) then there is no well-defined "entropy" of it, nor is there a well-defined temperature. If the system is rapidly changing so that it's not in equilibrium then there is also no well-defined temperature or entropy. Hope this clears up some of your questions about entropy!

From:

r6.livejournal.com

Oh, it’s the number of accessible states. For god’s sake, why didn’t books and web pages say that. I had the impression that it was the number of possible states.

From:

Thanks for the comment.

I briefly looked over the thread, and I noticed you asked "is entropy defined for a macroscopic particle". I think I know what you're asking here, but I'm not sure so correct me if I misinterpretted. I think what you're asking is: does entropy apply to classical systems which have no constituants small enough for quantum mechanics to play a role? And the answer is yes. Entropy is an entirely classical concept;

What I meant was "macroscopic" in the sense ping-pong balls: if we had a huge box in space (zero-gravity vacuum), with billions of ping-pong balls bouncing around in it, could we define entropy based on these observable states?
The problem for me is that it's not clear how you count states: at what level of detail do you look?

It seems to me that entropy is a "statistical" law, rather than a "physical" one: the 2nd law says that any system will tend to end up in the more probable set of states; and without knowledge of the system, one cannot bring it into a less probable set of states (these differences in probability being huge)

Does this make sense?

From:

Once it became possible to simulate a large number of interacting objects bouncing around exchanging energy, it became obvious that the entropy of a system is an objective quantity which refers to how many accessible states there are in the system.

So what's the solution to Maxwell's demon?

r6 and I believe that an all-knowing being could decrease the entropy of the box, while keeping his knowledge the same.

From:

ping-pong balls: Yes, though in practice the entropy from microscopic variation would be much greater still.

level of detail: See http://en.wikipedia.org/wiki/Entropy#Counting_of_microstates

From:

But the demon's knowledge must have a physical representation, and to run a full thermodynamic cycle it must reset its memory to be ready for the next cycle. This is a matter of erasing information, or increasing entropy.

i.e. this works only when the demon is outside the system being considered in thermodynamic terms.

From:

What I meant was "macroscopic" in the sense ping-pong balls: if we had a huge box in space (zero-gravity vacuum), with billions of ping-pong balls bouncing around in it, could we define entropy based on these observable states?

Yes, it's perfectly well defined. But only up to an additive constant which has to do with how finely grained things are. You can always add a constant to the amount of entropy in a system and it's not going to change the dynamics. As long as you're consistant as to what constant you add, it doesn't matter. Kind of like defining a "ground voltage". In quantum mechanics, there's a natural definition for what that constant should be, but that need not be essential to the theory of statistical mechanics; it can be seen just a convenience issue.

It seems to me that entropy is a "statistical" law, rather than a "physical" one: the 2nd law says that any system will tend to end up in the more probable set of states; and without knowledge of the system, one cannot bring it into a less probable set of states (these differences in probability being huge)

Yes, that's right... it's a law that comes entirely from statistics. There are some very basic physical assumptions that go into proving it, but other than that it's entirely a consequence of deductive reasoning, not something that just happens to be true in our physical world.

From:

darius has it exactly. In order to move one ball into a specified place you at least have to gain knowledge of that ball. The only option after that is to erase that knowledge or keep it. If you erase it, you must dissipate heat because that's an irreversible process. If you don't erase it than you end up needing a lot of storage in the long run.

From:

Imagine the Genius being outside of the box.
Since he knows everything about the deterministic system inside the box, his knowledge is the same before and after his trick of making heat flow the "wrong" way, so no resetting of the memory is necessary.

Is the relevant question "could he do it again to the same box?"? Obviously, you can't expect the box to go back to the exact same state as before, but I claim that he could take the box back to a "high entropy" state, and then the heat pump again. His memory is not increasing, and neither is it being erased: it's being updated at every "iteration", in a fully reversible process.

Would you say the entropy of the universe increased as a result of this? What could have happened in the Genius's brain to offset the massive entropy loss due to his heat trick?

From:

I've replied here

From:

Yes, that's right... it's a law that comes entirely from statistics. There are some very basic physical assumptions that go into proving it

which physical assumptions?

From:

Well, the main one is "ergodicity". There are a few other technical ones I think, but I wouldn't know off the top of my head. Only a few experts on the subject are concerned with what axioms are necessary to prove it, whereas most physicists are satisfied with it as long as it works. It wasn't proven rigorously until a long time after it was accepted as a standard law. And there are, I think, still debates on which axioms are the best ones to use.

Ergodicity is the main one though. Ergodicity means that you have to be dealing with a system which spends an equal amount of time in each "state" in the long run. Or, if its phase space is continuous... then the probability of it being in a particular region has to be proportionate to the volume of that region of phase space. ("phase space" is the 2n-dimensional space where position and momentum are the axes and n is the dimensionality of the regular "position" space--typically 3). It might make sense to say that ergodicity is just saying you need to define what a state is in a sane way. You can construct systems which don't satisfy ergodicity, but you could always say that that's just because you haven't labelled the states correctly.

From:

Hmmmm... that's a good question. I think the answer might be that in order to compute what's going to happen in the future for a complicated chaotic system you need to do a computation which increases entropy somehow. But it seems that the answer the this question would be pretty different for classical and quantum systems.

Maxwell's demon is something that I don't understand fully either, and I would definitely like to. Maybe we should both read this book on the subject. I particularly think Zurek has some important things to say about Maxwell's demon. He has a lot of papers on it and its combining it with quantum mechanics and information theory.

From:

That's the reason I quit physics: someone can go as far as you (and much further) not being able to answer such questions. No offense intended. It just shows that most physicists are not interested in logical consequences of / relationships between their theories. I get the impression that much reasoning physicists do is informal and ad-hoc, and I have no idea how easily they could justify them logically, if at all.

Maybe there is a school of physics where they really emphasize logical foundations? A place where people couldn't be happy unless they knew how to solve such paradoxes...

During sophomore year, it became clear that I wouldn't get what I wanted out of a physics education. It was quite a disappointment, but it seems I made the right decision.

From:

I think the answer might be that in order to compute what's going to happen in the future for a complicated chaotic system you need to do a computation which increases entropy somehow.

hm... I don't know much about reversible computation, but I argued above that you should be able to simulate the system forwards or backwards in time without erasing information, and without needing more memory.

It would be interesting if computation were under the same constraints as the physical system it simulates (e.g. if your computer contained a physical copy of the system it simulates), and irreversible computations corresponded to irreversible changes in the sytem (i.e. entropy increasing).

From:

Hm... maybe my comment wasn't called for. Some of these arguments seem quite involved.

It seems that physics curricula tend survey a bunch of ideas without analyzing their logical relationships to each other.

My approach is more to do learn a little bit, stop... check how it fits with everything.... and *then* proceed. This is the reason why I'm slower than most people: I'm always busy searching for contradictions (and incoherences). It's also the reason why my code is less buggy ;-)

what kind of book is this, btw? Maxwell's Demon's Greatest Hits? It seems each paper is from a different place.

From:

It would be interesting if computation were under the same constraints as the physical system it simulates (e.g. if your computer contained a physical copy of the system it simulates), and irreversible computations corresponded to irreversible changes in the sytem (i.e. entropy increasing).

Yes, that's what I'm saying is probably the case. But I don't know for sure, and the situation becomes more difficult when you ask the question of how quantum mechanics fits into the whole thing.

I have a pet "hypthesis" that in the many-worlds interpretation of quantum mechanics (the interpretation which I believe in) the total entropy in the multiverse should be constant (since everything is both reversible and non-chaotic). So far I haven't seen anyone addressing this question, but I plan on investigating it at some point if it hasn't already been dealt with. I wouldn't be too surprised either way, but I would really like to know the answer to this question.

From:

That's the reason I quit physics: someone can go as far as you (and much further) not being able to answer such questions.

When I took graduate statistical mechanics, the professor brought up Maxwell's demon one day and he said "I don't really understand this, but I'll explain it to the best of my ability." He explained it and then I asked him a few questions similar to the ones you asked. And he said "sorry, I just don't know. You'd have to ask an expert on it." The thing is, physics is a really large subject. Each person has a specialization. While they may understand most of the logical foundations of their specialization, nobody has the time to look into every question in infinite detail.

Maybe there is a school of physics where they really emphasize logical foundations? A place where people couldn't be happy unless they knew how to solve such paradoxes...

I'd say the school of physics that emphasizes logical foundations is "mathematics". Or philosophy, depending on which particular questions your asking about. (see my post on my view of the differences between these fields).

From:

My approach is more to do learn a little bit, stop... check how it fits with everything.... and *then* proceed. This is the reason why I'm slower than most people: I'm always busy searching for contradictions (and incoherences).

I work very much the same way. I'm by far the slowest physicsist (and thinker in general) that I've met. I try harder to understand the foundations of things before I'm happy with them, and as a consequence it takes me longer than most to learn a subject. Or sometimes just longer to admit that I've "learned it". Although I must confess that over this past year, my standards have been slipping a bit due to necessity. The sheer volume of stuff we have to know how to use has started to outweigh the possibility of understanding any of it fully. So my plan at this point is to go along with the way they want me to learn now, and then go back and learn it to my satisfaction when I get the chance. Hopefully, I won't just keep saying that and never go back. :)

From:

During sophomore year, it became clear that I wouldn't get what I wanted out of a physics education. It was quite a disappointment, but it seems I made the right decision.

Oh, one more thing I meant to say. The thing that attracts me most to physics is that it does pose questions which are so difficult to see the logical implications of. I enjoy sitting down and thinking about "hard" problems where the answer is not immediately obvious. Some of the problems in physics are so non-obvious that they've taken half a century or longer for people to answer. (Mawell's demon being one of those questions. I think it was posed back in the 1800's!). So I guess I've taken the opposite approach. Every time I see a physicist who I think doesn't really understand what he's doing, I see it as an encouragement that I have something to offer the field.

From:

I think your question is subtly different from Maxwell's Demon, who is supposed to extract useful work from the system, e.g. by pushing a piston against an external resistance. We can imagine two different idealizations of that interaction with the outside: either the demon has perfect knowledge/control of it -- it's microscopically reversible to the demon -- or it's a random interaction with a heat bath at some temperature. In the first case, to keep doing additional useful work we need an endless supply of zero-entropy resistance -- the demon must have an endless supply of prior information. In the second case, at the end of an expansion the demon no longer knows the exact state of the insides -- which I wrote about before.

This is still a bit handwavey, but I think that's the shape of the argument you're looking for -- that at some point you need to interact with an uncontrolled environment for this to matter. Physical law is such that information is conserved in time evolution (well, modulo quantum measurement); if the amount of information in the 'outside' is reduced, the amount in the 'inside' must be increased.

From:

That sounds like how Toffoli and Margolus put it (in Cellular Automata Machines -- that information is conserved and that's why we have thermodynamics. (I'm paraphrasing.) (Your point about the multiverse is something I've wondered about in this connection -- information doesn't seem to be conserved in the particular branch we're on after a measurement.)

From:

Wow, I'm surprised they mention that in a cellular automata book. I know Toffoli has done important work in quantum computing, though, which is somewhat closely tied to the modern many-worlds revival. I might be curious enough to pick up that book.

Here's the deal though... information is definintely conserved in the multiverse, that much is sure. The laws are all explicitly reversible. But the problem is, there are two different definitions of "reversible" and they're often confused. (Actually, I know of three different uses of the term in physics which all have very different meanings). Although I'd like to see entropy conserved in the multiverse too because it would make other stuff make more sense to me, I'm not sure it follows from the conservation of information. The old classical laws of physics were all reversible too, but entropy is not conserved there. I think the reason for that is that classical laws are chaotic. But I need to think about it more and read up more on these things to be sure.

As for thermodynamics coming from information conservation... I'd say that's overstating the case. Thermodynamics and statistical mechanics make sense for both classical and quantum systems... whether or not information is conserved. It might has a lot of bearing on how entropy works in our world, but the fact that you can still have a thermodynamics without it makes me cautious about making such claims.

From:

I wish I had the book ready-to-hand to relay it more accurately. It was a few remarks in the chapter on reversible CA, IIRC -- it left me wishing they'd spell it out more.

My take on it was, information from a god's-eye point of view is conserved: a 1-to-1 local update function in a CA, symplectic flows in classical mechanics, unitarity in quantum mechanics. But if you a flip a single bit in a reversible CA that's at all interesting, the trajectory rapidly diverges, and 'therefore' any imperfection in an agent within the system causes it to lose accurate information in its representation -- so the god's-eye conservation of information along with the interestingness of the dynamics means that agent's-eye information can decrease but not increase. (It can try to dump the lost information into parts of the system state it's not interested in, though.)

That seems like essentially the usual physics argument but from a bit of a more computer-sciencey point of view. I'm not a physicist, though.

That's interesting about Toffoli. The only physics paper of his I've read apparently claimed that the least-action principle actually amounts to having the universe waste the maximal amount of computation possible -- but I didn't understand it.

From:

My take on it was, information from a god's-eye point of view is conserved: a 1-to-1 local update function in a CA, symplectic flows in classical mechanics, unitarity in quantum mechanics. But if you a flip a single bit in a reversible CA that's at all interesting, the trajectory rapidly diverges, and 'therefore' any imperfection in an agent within the system causes it to lose accurate information in its representation -- so the god's-eye conservation of information along with the interestingness of the dynamics means that agent's-eye information can decrease but not increase. (It can try to dump the lost information into parts of the system state it's not interested in, though.)

Yes, that's exactly how it works. From the point of view of any observer in quantum mecahnics, information is always being destroyed. That's one of the main things I like the many-worlds-interpretation. It is the only interpretation where you can have a God's-eye view of the system where information is conserved. The illusion of the destruction of information is merely due to information flowing into different branches of the wavefunction which get causally isolated from each other due to decoherence (which is when it's useful to say the universe "splits" into disconnected pieces).

And because there's a close connection between information an entropy, this has led me to speculate that maybe entropy is conserved too. I'd like to see that turn out true, but it's only a guess. In thermodynamics, whenever things are "reversible" then entropy is conserved. But the usage of reversible in that context is a little different from the usage in the context of information. I don't completely understand the connections between the two, but I haven't had a chance to look into it more.

That's interesting about Toffoli. The only physics paper of his I've read apparently claimed that the least-action principle actually amounts to having the universe waste the maximal amount of computation possible -- but I didn't understand it.

There's a universal gate in quantum computing called the Toffoli gate which has great importance. Maybe they borrowed the name from elsewhere, but I assumed he had come up with it in the context of thinking about quantum computing. I don't know about that least-action comment, that sounds very strange!

From: