Second attempt. If they keep disappearing, I will have to give up at some point, but fortunately I had this one from yesterday still under the Crtl-V key...

> And how will you know that there isn't some other potential flaw that you haven't thought of? 

Well, 'exclude' in this context should be taken to mean 'beyond reasonable doubt', and not in any mathematical sense. The whole concept of piece value has no mathematical meaning to begin with. It is just a heuristic for estimating your winning chances. The point is that the empirical method is in general quite insensitive to poor play, and that the effects one might not think of (and even those we do think of) are tiny to begin with. Joker will stupidly 'gain' a Knight for his two Pawns in KNPPKN, thinking that it ups his score from +2 to +3. Because I never told it KNK is a hard 0, rather than +3. But the number of games where it ends in KNK or KBK in practice is very small, less than 1% even when you play it against engines of similar strength that do have this knowledge, which is why I never bothered to correct this behavior. Even such a major blunder hardly affects the overall score.

And, even more important, the method uses self-play. When both sides are unaware of KNK being a draw, it happens even less frequently, because the side that is behind will foolishly resist the opponent 'driving up' his advantage to as much as a full Knight. So somtimes the side with the Knight will discover to his surprise that he cannot win in a +3 position because it is KNK. But at other times he will be lucky in a Knight ending, because his opponent will fail to recognize he could draw by a Knight sac in KNPKN, and will win rather than draw because the opponent does not get a second opportunity by the time they can see the promotion, and count the P as Q. The lack of knowledge always works both ways.

This is why I would be very surprised to see a large effect of including mate-potential knowledge. And that holds even more for more subtle effects.

> Should we dismiss a bunch of experts that have made no mistake you can point out simply because we also can't point out a mistake that your engine has made when it gives a different result?

I am a bit skeptical when I hear the word 'expert'. How many games would these experts have played? My computer does thousands in a few days. So one obvious mistake I think I can point out for the experts is: 'not enough games to get a representative sampling of possible situations'. Furthermore, in human games it is difficult to have players of equal strength. Even if their overall strength is equal, they might have different weaknesses. A player that is pretty adept with Commoners will use them to spring unpleasant surprises on his opponent who isn't,and doesn't see the danger coming. So that player could mistakingly get the impression that the Commoners are pretty strong, while in fact the only thing he is proving is that his opponent is 'Commoner-blind'. It is just very hard to get the cancellation of weaknesses that is automatic in self-play.

In addition computers do not suffer from systematic oversight,as they rely on exhaustive search. People rely on ideas, and the ideas can easily be wrong or incomplete.

> Alternately: there are significant differences between the ways humans and computers play chess, so theoretically some pieces could have a different effective value in human vs. human games than in computer vs. computer...

Yes, this is a possibility. But if the play isn't very poor, the empirical results turn out to be surprisingly independent of the strength of the players. So I would be very surprised if it did depend very much on the strength of the strategic or tactical contributions to this overall strength. Which seems the major differencce between computer and human play. It is a case in point that computers tend to value the Queen higher than humans do. But even statistics from human games shows that the Queen is more valuable than the humans think (9.75 rather than 9.5). So apparently humans have a tendency to underestimate the value the Queen has to themselves! But that doesn't fully disguise the fact that it is stronger.